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I.      INTRODUCTION 

The  reliability  of  a  particular  system  or  piece  of  equipment  is  one  of  the  most  im- 
portant factors  in  assessing  its  true  value.  A  system  with  tremendous  capabilities  that 
spends  an  inordinate  amount  of  time  in  the  maintenance  facility  is  usually  less  desirable 
than  an  alternate  system  with  relatively  moderate  capabilities  that  has  a  much  higher 
availability.  Therefore,  accurately  measuring  the  reliability  of  a  proposed  system  during 
the  test  and  design  phases  is  an  integral  part  of  the  materiel  acquisition  process.  Tradi- 
tional reliability  estimators  rely  upon  conducting  a  sufficient  number  of  tests  or  exper- 
iments so  that  a  statistically  valid  point  estimate  or  confidence  interval  can  be 
established.  These  traditional  estimates  assume  a  constant  or  fixed  reliability  during  any 
particular  design  phase.  Since  reliability  will  almost  certainly  fluctuate  from  phase  to 
phase  of  the  acquisition  process,  methods  are  needed  that  track  reliability  as  it  changes. 
Additionally,  systems  being  produced  presently  are  more  complex  and  claim  a  much 
higher  reliability  than  those  produced  just  a  few  years  ago.  Verifying  high  reliability  in- 
herently requires  significantly  more  testing  than  does  verifying  relatively  moderate  reli- 
ability since  items  are  tested  until  failure.  Therefore,  all  test  data  should  be  utilized  to 
the  utmost. 

The  process  of  testing  and  evaluating  newly  designed  pieces  of  equipment  is  very 
costly  in  terms  of  both  time  and  money.  Typically,  the  test  engineer  must  face  limitations 
in  both  of  these  resources  and  occasionally  must  compromise  on  either  the  extensiveness 
of  the  test  design  or  the  number  of  trials  to  be  run  or  both.  It  may  be  assumed  that  the 
constraints  on  time  and  money  will  not  diminish  in  the  near  future  --  if  anything,  these 
constraints  will  become  tighter.  Therefore,  methods  that  are  able  to  produce  accurate 
estimates  of  system  reliability  for  a  smaller  investment  of  resources  are  very  desirable. 
Reliability  growth  models  are  one  such  method. 

Reliability  growth  models  make  use  of  all  available  test  data.  Results  from  previous 
phases  of  design  are  combined  with  current  test  data  so  that  a  pattern  of  reliability  is 
established.  These  models  often  make  use  of  a  procedure  known  as  failure  discounting. 
Failure  discounting  involves  removing  fractions  of  previous  failures  in  order  to  make 
allowance  for  reliability  improvement  as  the  system  evolves.  The  reliability  pattern  es- 
tablished by  a  reliability  growth  model  becomes  the  basis  for  producing  estimates  of  the 
actual  system  reliability. 


There  are  two  types  of  reliability  growth  models  --  discrete  and  continuous.  A 
continuous  model  is  based  on  the  time  until  failure  of  the  system  under  consideration. 
Obviously,  a  number  of  distributional  assumptions  are  necessary  in  a  model  of  this  type. 
Discrete  reliability  growth  models  are  employed  when  test  data  references  the  number 
of  system  tests  and  failures  in  a  particular  design  phase.  In  order  to  use  a  discrete  reli- 
ability growth  model  one  must  be  able  to  classify  a  test  result  as  either  a  failure  or  a 
success.  In  other  words,  the  test  data  must  be  attribute  data.  In  this  thesis,  situations 
where  test  parameters  may  classify  a  trial  as  a  partial  success  or  a  partial  failure  are  not 
allowed. 

This  thesis  addresses  discrete  reliability  growth  models.  The  major  objective  is  to 
analyze  the  capabilities  and  limitations  of  two  such  models.  Two  different  methods  of 
discounting  previous  failures  are  also  evaluated.  A  previously  designed  simulation  pro- 
gram [Ref.  1:  pp.  36-46]  is  expanded  or  modified  as  necessary  in  order  to  more  fully  de- 
velop the  two  discrete  reliability  growth  models  being  examined.  The  original  simulation 
uses  actual  reliabilities  and  Monte  Carlo  techniques  in  order  to  generate  a  random  reli- 
ability growth  pattern.  The  test  data  generated  by  this  growth  pattern  is  then  used  in  the 
reliability  growth  models  in  order  to  produce  estimates  of  system  reliability.  A  variety 
of  different  reliability  growth  patterns  and  a  broad  spectrum  of  failure  discounting  pa- 
rameters are  systematically  evaluated.  At  each  phase,  the  estimate  of  system  reliability 
is  compared  to  the  actual  reliability.  The  applicability  of  the  two  discrete  reliability 
growth  models,  to  include  limitations  and  capabilities  is  also  addressed. 

The  following  chapter  describes  each  of  the  two  discrete  reliability  growth  models 
being  considered.  An  explanation  of  the  two  failure  discounting  methods  is  also  offered. 
Additionally,  work  done  in  this  area  previous  to  this  analysis  is  summarized.  Chapter  III 
discusses  the  methodology  used  co  conduct  the  analysis,  to  include  the  computer  simu- 
lation. Chapter  III  also  describes  the  various  different  growth  patterns  that  were  ana- 
lyzed. Chapter  IV  presents  the  results  of  the  analysis.  Chapter  V  details  conclusions  and 
recommendations. 


II.      DISCOUNTING  PROCEDURES  AND  DISCRETE  GROWTH 

MODELS 

A.  BACKGROUND 

In  this  chapter,  two  discrete  reliability  growth  models  are  described.  Two  separate 
techniques  of  discounting  failures  are  also  explained.  Both  methods  remove  a  fraction 
of  a  previous  failure  when  the  system  successfully  completes  a  trial  without  a  reoccur- 
rence of  the  particular  failure  cause.  Reliability  is  defined  as  the  probability  of  the  sys- 
tem successfully  completing  a  single  trial.  The  exact  definition  of  a  test  trial  is  much 
more  difficult  to  fix  since,  typically,  the  constitution  of  a  trial  is  the  responsibility  of  the 
testing  agency.  Normally,  a  trial  may  be  considered  as  the  exercising  of  the  particular 
system  in  a  manner  consistent  with  its  purpose.  If  a  weapons  system  is  being  evaluated, 
for  example,  then  a  trial  could  consist  merely  of  a  single  attempt  at  target  acquisition 
or  proceed  through  target  acquisition  to  an  attempt  at  target  destruction.  The  precise 
definition  of  a  trial  depends  in  large  measure  on  the  purpose  or  goal  of  the  test.  For  the 
purpose  of  this  thesis  and  in  order  to  utilize  discrete  reliability  growth  models,  a  trial 
must  be  defined  so  that  it  is  discrete  and  can  be  evaluated  as  either  a  success  or  failure. 
Partial  successes  are  not  allowed. 

A  testing  phase  may  be  defined  as  a  number  of  trials,  one  or  more,  during  which 
the  configuration  of  the  system  is  unchanged.  Therefore,  during  a  testing  phase  the  ac- 
tual system  reliability  remains  unchanged.  A  test  phase  may  consist  of  one  or  more 
system  failures.  If  improvements  or  changes  to  the  system  are  effected  after  each  system 
failure  (a  test-fix-test  scenario)  then  a  phase  would  consist  of  one  system  failure  and  all 
of  the  successful  trials  leading  up  to  that  failure.  Alternatively,  the  test  design  may  dic- 
tate that  testing  be  continued  until  a  certain  number  of  failures  occur.  Under  this  sce- 
nario (test-find-test)  the  cause  of  system  failure  is  identified  but  system  configuration 
remains  unchanged  until  the  pre-designated  failure  occurs.  In  general,  a  phase  will  en- 
compass all  of  the  trials  between  system  configuration  changes. 

B.  FAILURE  DISCOUNTING 

Testing  conducted  during  the  initial  design  stages  of  a  particular  system  often  indi- 
cates low  reliability.  Generally,  weaknesses  in  the  configuration  of  the  system  or  defects 
in  the  quality  of  its  components  cause  system  failure.    Test  designs  are  established  so 


that  the  cause  for  these  failures  can  be  identified  and  corrected.  Theoretically,  then,  as 
a  weakness  or  a  defect  is  identified  and,  hopefully,  corrected  the  probability  of  that 
particular  weakness  or  defect  reoccurring  should  be  reduced.  This  reduction  in  the 
probability  of  occurrence  of  a  certain  failure  cause  leads  to  improved  system  reliability. 
This  concept  is  fully  utilized  in  failure  discounting. 

In  order  to  effectively  discount  previous  failures  it  is  critical  that  the  cause  of  the 
failure  be  properly  identified.  The  level  of  detail  that  one  wishes  to  ascribe  to  this  iden- 
tification process  is  dependent  upon  the  type  of  system  being  evaluated  and  the  purpose 
of  the  test.  If  a  complex  system  is  being  evaluated  then  a  failure  cause  may  be  failure 
of  a  certain  component  or  sub-component.  The  precise  element  that  caused  system 
failure  is  not  critical  but  the  ability  to  assign  a  failure  cause  to  each  system  failure  is. 
Correctly  determining  failure  cause  is  very  difficult,  particularly  when  dealing  with  com- 
plex systems.  Therefore,  it  is  conceivable  that  design  changes  do  not  improve  system 
reliability  --  in  fact,  these  changes  may  even  degrade  reliability.  To  apply  the  failure 
discounting  procedures  described  below,  one  must  be  able  to  assign  a  failure  cause  to 
every  system  failure  even  though  it  may  be  done  erroneously. 
1.      Standard  or  Straight  Percent  Failure  Discounting 

The  standard  discounting  method  will  reduce  a  previous  failure  by  a  fixed 
amount  after  a  predesignated  number  of  trials  have  been  successfully  conducted  without 
a  reoccurrence  of  a  failure  due  to  the  same  cause.  This  method  has  been  detailed  fully 
in  previous  work  [Ref.  1:  pp.  14-17]  so  only  a  brief  description  and  an  example  of  its 
application  will  be  included  here. 

To  employ  the  standard  discounting  method  one  must  specify  two  parameters 
-  the  fraction  of  a  failure  to  be  removed,  F,  and  the  discount  interval  or  the  number  of 
successful  trials,  I,  that  must  occur  before  application  of  the  discount.  For  the  purpose 
of  this  paper  and  to  use  the  simulation  described  herein,  these  parameters  are  specified 
at  the  outset  and  remain  constant  throughout  the  test.  The  number  of  successful  trials 
since  the  occurrence  of  the  failure  is  referred  to  as  T.  These  three  values,  F,  I,  and  T 
are  then  used  to  discount  previous  failures  at  the  end  of  each  phase. 

Mathematically,  the  standard  discounting  method  is  given  by  Equation  2.1: 

DISCOUNTED  FAILURE  =  {\-F),NT{Jr)  (2.1) 

The  term  INT  in  the  above  equation  refers  to  the  integer  portion  of  the  argument 

T 

(  —r )  •   As  such,  if  the  number  of  successful  trials,  since  the  failure  cause  last  occurred, 


T 
T,  is  less  than  the  number  of  trials  required  by  the  discount  interval,  I,  then  INK  —  ) 

will  be  zero  thus  causing  no  discounting  to  be  applied.  Additionally,  should  a  failure 
cause  reoccur  then  the  value  of  T  is  returned  to  zero  thus  removing  any  previous  dis- 
counting that  may  have  been  applied.  This  ability,  in  effect,  acts  as  a  penalty  factor. 
If  a  failure  cause  should  reoccur  then  previous  design  changes  may  not  have  been  ef- 
fective and  thus  any  prior  failure  discounting  could  be  unwarranted.  The  rate  and 
amount  of  failure  discounting  can  be  controlled  by  altering  F  and  I.  If  the  system  under 
evaluation  is  relatively  simple  and  a  proven  failure  identification  process  is  in  place,  then 
a  large  discount  to  system  failures  may  be  warranted.  This  will  result  in  a  prediction  of 
rapid  reliability  growth. 


Table  1.      STANDARD  DISCOUNT  METHOD  EXAMPLE 

PHASE 

TRIAL 

TOTAL 
FAILURES 

CAUSE 

X 

Y 

1 

1 

0 

S 

s 

I 

-) 

1 

F 

S 

2 

3 

1 

S 

S 

2 

4 

1 

S 

S 

2 

5 

2 

S 

F 

3 

6 

2 

s 

S 

3 

7 

2 

s 

S 

3 

8 

2 

s 

S 

i 
j 

9 

2 

s 

S 

3 

10 

2 

s 

S 

5 

11 

2 

s 

s 

3 

12 

3 

F 

S 

A  small  example  of  the  application  of  the  standard  discount  method  will  aid  in 
clarifying  its  abilities.  Consider  the  data  in  Table  1.  This  test  data  is  purely  fictitious 
and  is  intended  solely  to  illustrate  the  use  of  the  standard  discount  method.  We  will 
assume  that  improvements  or  repairs  are  effected  after  each  failure  so  actual  system  re- 
liability is  not  constant. 

Table  1  represents  three  phases  of  a  test-fix-test  scenario  where  two  failure 
causes,  X  and  Y,  have  been  identified.  It  should  be  noted  that  under  a  test-fix-test  plan, 


a  phase  will  terminate  with  system  failure.  The  result  of  each  trial  is  listed  as  either  a 
success  or  failure  for  a  particular  failure  cause. 

The  two  parameters  necessary  to  employ  the  standard  discount  method  are  the 
fraction  of  failure  to  be  removed,  F,  and  the  discount  interval,  I.  Assume  that  F  =  0.50 
and  1  =  3.  System  failure  number  two,  attributed  to  failure  cause  Y,  terminates  phase 
two.  At  that  point  the  failure  due  to  cause  X  that  terminated  the  first  phase  has  had 
three  successive  successful  trials  so  failure  discounting  can  be  applied: 

ADJUSTED  FAILURE  =  (1  -  0.50)(T}  =  0.50 

Thus,  for  reliability  computations  at  the  conclusion  of  the  second  phase,  the  failure  that 
occurred  during  phase  1  is  only  counted  as  one-half  of  a  failure. 

Phase  three  terminates  with  a  failure  due  to  cause  X.  The  value  for  T  then  is 
set  to  zero  and  thus  when  applied  to  the  failure  that  occurred  in  phase  one,  full  value  is 
restored.  At  the  end  of  phase  two  this  failure  had  been  discounted  to  0.50  but,  because 
the  failure  cause  reoccurred,  this  discount  is  viewed  as  unwarranted  and  the  failure  is 
returned  to  one.  However,  at  the  end  of  phase  three  failure  cause  Y  has  had  seven  suc- 
cessful trials.   Therefore,  discounting  is  applied: 

ADJUSTED  FAILURE  =  (1  -  0.50)IN7lT)  =  (0.50)2  =  0.25 

Therefore,  when  computing  reliability  estimates  at  the  end  of  phase  three,  only  2.25 
system  failures  (failure  number  one  plus  failure  number  three  plus  the  discounted  value 
of  failure  number  two)  are  considered  instead  of  the  three  failures  that  actually  occurred. 

The  standard  discount  method  described  is  very  flexible  because  of  its  two  pa- 
rameters. Setting  values  for  F  and  I  does,  however,  require  a  good  deal  of  professional 
judgement  and  expertise.  To  avoid  conflicts  in  setting  such  values  David  K.  Lloyd  pro- 
posed an  alternate  method  of  discounting  failures  [Ref.  2].  The  Lloyd  method  is  de- 
scribed in  the  next  section. 

2.      Lloyd  Failure  Discounting 

Discounting  previous  failures  using  the  Lloyd  discounting  method  is  based  on 
the  premise  that  there  should  be  some  sort  of  statistical  basis  for  determining  how  much 
a  previous  failure  should  be  discounted.  Lloyd  ofiers  the  upper  confidence  limit  of  the 
probability  that  a  failure  cause  will  occur  again  as  the  discounted  value  of  its  respective 


failure.  The  confidence  limits  for  this  probability  are  based  on  the  number  of  successful 
trials  since  the  failure  cause  last  occurred.  Under  a  test-fix-test  scenario  this  is  equivalent 
to  the  number  of  successful  trials  since  the  last  failure.  This  number  was  defined  in  the 
description  of  the  standard  discount  method  as  T.  In  order  to  implement  Lloyd  failure 
discounting,  one  must  set  the  level  of  confidence  desired  for  the  confidence  bound.  Since 
the  method  is  applied  on  every  trial  there  is  no  requirement  to  specify  an  interval,  al- 
though as  will  be  explained  in  a  later  chapter,  specifying  an  interval  for  the  Lloyd 
method  can  be  done  and  may  be  advantageous  in  some  situations. 

The  Lloyd  discount  method  may  be  mathematically  expressed  as  follows: 


ADJUSTED  FAILURE  =  1  -  (1  -  CL)~   i/T>0 


(2.2) 


In  the  above  equation  CI  is  defined  as  the  level  of  confidence  desired  and  T  is  the 
number  of  successful  trials.  If  T  has  a  value  of  zero  then  the  value  of  the  adjusted  failure 
is  set  to  one.  Setting  the  value  of  the  adjusted  failure  equal  to  one  when  T  equals  zero 
gives  the  Lloyd  method  the  ability  to  restore  all  of  the  value  of  a  discounted  failure 
should  its  failure  cause  reoccur. 


Table  2.     LLOYD  DISCOUNT  METHOD  EXAMPLE 


PHASE 

TRIAL 

TOTAL 
FAILURES 

FAILURE 
CAUSE 

ADJUSTED 
FAILURES 

X 

Y 
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1 
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1 

F 
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1.000 

2 

3 

1 

S 

s 

0.900 

2 

4 

1 

S 

s 

0.684 

-» 

5 

2 

S 

F 

0.536  +   1.000  =   1.536 

3 

6 

t 

S 

S 

0.438  4-  0.900  =   1.338 

"> 

j 

7 

-> 

S 

S 

0.369  +  0.684  =   1.053 

3 

S 

-) 

S 

S 

0.319  +  0.536  =  0.S55 

9 

•> 

s 

S 

0.280  +  0.438  =  0.71  S 

i 
J 

10 

2 

s 

S 

0.250  +  0.369  =  0.619 

-y 
J 

11 

•> 

s 

S 

0.226  +  0.319  =  0.545 

-* 
J 

12 

^ 
j 

F 

S 

2.000  +  0.2S0  =  2.2S«> 

Table  2  illustrates  the  use  of  the  Lloyd  method.  The  test  data  is  the  same  data 
used  to  demonstrate  the  standard  discount  method.  Since  the  Lloyd  method  is  applied 
on  every  trial  a  fourth  column  has  been  added  to  the  table  reflecting  the  current  value 
of  the  adjusted  failure.  Let  us  assume  that  the  confidence  interval  desired  in  this  case 
was  .90. 

If  reliability  is  computed  at  the  completion  of  the  third  phase  then  2.28  system 
failures  would  be  used  under  the  Lloyd  discounting  method  as  opposed  to  the  actual 
value  of  three  that  occurred.  The  last  column  of  the  table  indicates  the  speed  that  failures 
become  discounted  with  the  Lloyd  method. 

The  two  discrete  reliability  growth  models  that  will  be  examined  are  described 
in  the  next  section.  Each  model  makes  use  of  discounted  failure  data  and  each  has 
demonstrated,  through  simulation,  the  ability  to  estimate  constant  and  concavely  in- 
creasing reliability  with  reasonable  accuracy. 

C.      DISCRETE  RELIABILITY  GROWTH  MODELS 

Both  of  the  discrete  reliability  growth  models  to  be  presented  assume  a  constant 
system  reliability  within  a  phase.  The  first  model  to  be  examined  will  be  the  Maximum 
Likelihood  Estimate  with  Failure  Discounting  (MLEFD).  Tliis  model  is  merely  a  deriv- 
ative of  the  conventional  single  phase  maximum  likelihood  estimate.  The  single  phase 
maximum  likelihood  estimate  considers  only  test  data  generated  during  the  current 
phase.  The  MLEFD,  conversely,  is  cumulative  in  that  it  considers  all  of  the  available 
test  data.  The  test  data  from  early  phases  during  which  the  actual  system  reliability  was 
different  requires  adjustment.  Thus,  the  use  of  failure  discounting  techniques  provides 
a  method  by  which  failures  occurring  early  in  the  development  process  can  be  assim- 
ilated into  current  reliability  estimates.  The  second  model  is  a  regression  model  based 
on  an  exponential  single  phase  reliability  estimate.  It  estimates  reliability  for  each  phase 
using  an  exponential  model  and  then  performs  a  linear  regression  on  these  estimates  to 
obtain  a  current  estimate.  The  simulation  program  that  was  written  to  evaluate  these 
two  models  was  designed  so  that  failure  discounting  may  or  may  not  be  invoked  with 
either  model. 

Before  offering  a  general  description  of  the  two  models  it  is  important  at  this 
juncture  to  digress  for  a  moment  and  discuss  the  exact  methodology  that  will  be  used  to 
incorporate  discounted  failures  into  the  various  reliability  estimates.  As  was  seen  in  the 
section  prior,  discounted  or  adjusted  failures  are  rarely  if  ever  integer- valued.  While  this 
does  not  present  a  problem  for  the  MLEFD  reliability  growth  model  it  does  pose  an 


obstacle  to  using  the  exponential  regression  model.  The  convention  devised  in  Captain 
Drake's  thesis  [Ref.  1:  p.  23]  will  be  continued  in  this  paper.  The  number  of  trials  up  to 
and  including  a  particular  failure  will  be  divided  by  the  adjusted  failure  so  that  an  ad- 
justed number  of  trials  is  computed.  The  adjusted  failure  value  is  then  returned  to  one 
from  its  previous  fractional  value.  The  formula  for  computing  adjusted  trials  is  then: 

ADJUSTED  TRIALS  =     j^MLIEM^.  (2.3) 

The  appeal  of  this  method  is  that  the  result  is  an  integer  number  of  failures  while  main- 
taining a  constant  ratio  of  number  of  failures  to  number  of  trials.  While  the  adjusted 
number  of  trials  may  not  be  integer-valued  they  may  be  adjusted  to  integers,  if  necessary, 
through  rounding  with  a  much  smaller  effect  on  reliability  computations  than  rounding 
adjusted  failures. 

If  a  failure  had  been  discounted  (using  either  discounting  method)  from  one  to  .25, 
for  example,  and  the  number  of  successful  trials  since  the  failure  occurred  was  ten  then 
the  ratio  of  failures  to  successes  would  be  .25  to  ten.  Using  the  method  described  above, 
the  adjusted  failure  and  actual  trials  are  divided  by  the  adjusted  failure  yielding  a  failure 
to  success  ratio  of  one  to  forty.  Thus,  the  number  of  trials  has  been  adjusted  from  ten 
to  forty  and  the  number  of  failures  has  been  returned  to  one  from  .25.  Since  the  ratio 
of  failures  to  successes  remains  the  same  reliability  computations  will  not  be  affected. 
Both  of  the  models  described  below  will  make  use  of  this  method  in  computing  reliability 
estimates. 

1.      Maximum  Likelihood  Estimate  with  Failure  Discounting 

The  traditional  estimate  of  system  reliability  is  the  maximum  likelihood  esti- 
mate. Letting  R  denote  the  estimate  of  reliability,  the  maximum  likelihood  estimate  may 
be  expressed  as  follows: 

£  .       TOTAL  TRIA LS  -  TO TA L  FA IL URES 


TOTAL  TRIALS 
or  alternatively: 

SUCCESSFUL  TRIALS 


R  = 


TOTAL  TRIALS 


As  stated  previously,  the  maximum  likelihood  estimate  assumes  a  constant  re- 
liability between  phases.  It  also  requires  a  large  number  of  trials  to  accurately  estimate 


the  underlying  system  reliability.  Failure  discounting  is  used  to  make  previous  test  data 
compatible  with  current  results  and  thus  increase  the  total  number  of  trials  available. 
However,  since  the  actual  system  reliability  is  unknown  it  becomes  a  very  non- trivial 
task  to  select  the  appropriate  discounting  method  and  the  correct  parameters. 

When  previous  test  data  is  made  available  through  the  use  of  a  discounting 
routine  the  equation  for  estimating  the  system  reliability  becomes: 

£         TOTAL  TRIALS  -  TOTAL  ADJUSTED  FAILURES 

TOTAL  TRIALS 

or  if  the  convention  of  restoring  the  number  of  failures  to  integer  values  is  adopted 

£         TOTAL  ADJUSTED  TRIALS  -  TOTAL  FAILURES  (2  .. 

TOTAL  ADJUSTED  TRIALS  ['} 

To  utilize  the  MLEFD  reliability  growth  model  one  first  records  all  of  the  test 
data  from  the  current  phase.  The  discounting  method  to  be  used  is  then  performed  on 
the  previous  test  data  and  an  adjusted  failure  value  is  computed  for  each  previous  failure. 
The  adjusted  number  of  trials  for  each  failure  is  then  computed  using  Equation  2.3.  The 
estimate  of  reliability  for  the  current  phase  is  then  calculated  using  Equation  2.4  above. 

One  deficiency  of  the  maximum  likelihood  estimate  is  that,  when  testing  is  ter- 
minated after  a  fixed  number  of  failures,  the  MLE  is  inherently  a  biased  estimator  of  the 
actual  reliability.  The  expected  value  of  the  maximum  likelihood  estimate  has  been  de- 
rived in  previous  work  [Ref.  3:  p. 34].    It  may  be  expressed  mathematically  as  follows: 

E[K]  =  1  +  (~)ln(l-£)  -    \n{[-R)  (2.5) 

K 

where  R  represents  the  actual  reliability  of  the  system.  The  bias,  B{R),  in  the  estimate 
may  be  calculated  using  the  formula: 

B{R)  =  £[£]  -  R 

Table  3  below  depicts  the  performance  of  the  maximum  likelihood  estimate  for 
different  values  of  actual  reliability,  R.  The  bias  is  inherent  to  the  estimate  in  that  it  will 
be  present  regardless  of  the  validity  of  the  assumption  of  constant  phase  reliability. 

The  use  of  failure  discounting  can  offset  the  effect  of  bias  as  well  as  handle 
changing  system  reliability.  However,  this  puts  a  lot  of  additional  weight  on  the  ability 
to  correctly  select  the  best  method  and  choose  the  proper  parameters.  Alternatively,  if 
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an  estimator  of  system  reliability  had  to  be  biased  one  would  want  it  to  be  conservatively 
biased  as  is  the  MLE.  Underestimating  actual  svstem  reliabilitv  is  normallv  less  costlv 
than  overestimating.  However,  a  less  biased  estimate  would  be  beneficial. 

Table  3.     INHERENT  BIAS  IN  THE  MAXIMUM  LIKELIHOOD  ESTIMATE 


RELIABILITY 

E[R] 

B(R) 

0.50 

0.307 

-0.193 

0.60 

0.389 

-0.211 

0.70 

0.484 

-0.216 

0.80 

0.598 

-0.202 

0.90 

0.744 

-0.156 

0.95 

0.842 

-0.108 

0.97 

0.S92 

-0.078 

0.99 

0.953 

-0.037 

2.      Exponential  Regression  Estimate 

The  exponential  regression  reliability  growth  model  was  developed  by  H. 
ChernofT  and  W.M.  Woods  and  based  on  the  exponential  single  phase  estimate  [Ref. 
4].  The  derivation  of  this  model  has  been  developed  fully  in  previous  work  [Ref.  5:  pp. 
3-5]  so  for  the  purposes  of  this  paper  a  general  summary  of  results  will  be  offered. 

The  basic  form  of  the  exponential  single  phase  estimate  for  a  phase  k  i.e.,  after 
the  kth  improvement  or  alteration  has  been  made,  is  stated  in  Equation  2.6: 


Ru  =  \-e 


{-Yk) 


(2.6) 


where  Rk  is  the  reliability  estimate  and  the  coefficient  of  the  exponential  term,  Y,  can 
be  calculated  as  follows: 


Yy   =     I  +4" +.4-+. ..+ 


1 


(*A   "  1) 


if  Xk  >  2 


where  Xk  is  the  number  of  trials  up  to  and  including  the  first  failure  in  phase  k.  If  Xk 
equals  one  then  Yk  is  defined  equal  to  zero.  It  should  be  noted  that  tills  estimate  was 
developed  for  the  special  case  of  testing  until  the  first  failure  is  encountered.  If  system 
failure  occurs  on  the  first  trial  (Xk  =1)  then  Yk  will  equal  zero  and  system  reliability  will 
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be  estimated  as  1  -  e°  =  1-1  =  0.  If  failure  occurs  on  the  third  trial,  then  the  expo- 
nential single  phase  estimate  would  yield  I  —  e-<1+7>  =  0.777. 

In  order  to  evaluate  the  properties  of  the  exponential  single  phase  estimate  a 
small  simulation  was  designed.  Geometric  test  data  with  actual  reliability  R  were  gener- 
ated using  a  transformation  of  a  standard  Uniform(0,l)  random  number.  Ten  thousand 
sets  of  geometric  data  were  generated  for  each  value  of  R.  Table  4  represents  the  results 
of  this  simulation.  Also  included  in  the  table  is  a  simulation  of  the  maximum  likelihood 

A 

estimate,  (R),  using  the  same  test  data. 

The  values  generated  for  the  mean  of  the  maximum  likelihood  estimate,  R  , 
conform  fairly  well  with  the  theoretical  results,  thus  the  simulation  itself  is  accurate.  The 
values  generated  for  the  exponential  estimate,  R  ,  while  still  underestimating  the  actual 
reliability,  are  clearly  less  conservatively  biased  than  the  maximum  likelihood  estimates. 
The  sample  variance  and  mean  square  error  of  each  estimate  is  also  represented  in  the 
table.  The  mean  square  error  of  each  estimate  is  the  sum  of  the  variance  and  the  square 
of  the  bias.  It  can  be  seen  that  as  the  actual  reliability  exceeded  .70  the  mean  square 
error  for  the  exponential  estimate  became  less  than  the  mean  square  error  of  the  maxi- 
mum likelihood  estimate.  Therefore,  the  advantage  of  a  lower  variance  for  the  maximum 
likelihood  estimate  at  values  of  actual  reliability  below  .90  is  offset  by  the  greater  bias. 


Table  4.     PROPERTIES  OF  THE  EXPONENTIAL  RELIABILITY  ESTIMATE 


R 

ELK] 

Maximum  Likelihood  Estimate 

Exponential  Estimate 

R 

Var(R) 

MSE(R) 

R 

Var(R) 

MSE{R) 

0.50 

0.307 

0.302 

0.101 

0.141 

0.357 

0.137 

0.157 

0.60 

0.389 

0.387 

0.111 

0.157 

0.451 

0.143 

0.166 

0.70 

0.484 

0.480 

0.115 

0.163 

0.547 

0.140 

0.163 

0.80 

0.598 

0.595 

0.107 

0.149 

0.661 

0.121 

0.140 

0.90 

0.744 

0.743 

0.079 

0.103 

0.797 

0.080 

0.090 

0.95 

0.842 

0.842 

0.050 

0.062 

0.881 

0.047 

0.052 

0.97 

0.892 

0.892 

0.034 

0.040 

0.921 

0.031 

0.033 

0.99 

0.953 

0.954 

0.014 

0.015 

0.968 

0.012 

0.012 
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The  exponential  regression  reliability  growth  model  developed  by  Chernoll'and 
Woods  uses  the  technique  of  linear  regression  to  estimate  the  coefficient  of  the  expo- 
nential term.  The  model  can  be  expressed  mathematically  as  follows: 

Rk  =   1  -  e-{^k)  (2.7) 

In  the  equation  above  k  denotes  the  testing  phase  being  used  to  compute  the 
system  reliability,  a  and  /?  are  computed  at  the  end  of  each  phase  using  the  techniques 
of  linear  regression  thus  yielding  an  estimate  of  system  reliability.  The  exact  derivation 
of  the  linear  regression  formula  for  a  and  /?  is  detailed  in  other  sources  [Ref.  3:  p.  3]  and 
is  provided  at  Appendix  A  for  the  interested  reader. 

This  model  requires  that  both  failures  and  trials  be  integers.  Therefore,  when 
adjusted  trials  are  computed  using  Equation  2.3,  the  value  obtained  must  be  rounded  to 
the  nearest  integer.  Since  linear  regression  is  employed,  this  model  has  the  ability  to 
track  changing  reliability  without  the  use  of  failure  discounting. 

D.      SUMMARY  OF  PREVIOUS  WORK 

Analysis  of  reliability  growth  models  has  been  a  continuing  process,  and  many 
advances  have  been  made  in  determing  their  properties  [Refs.  6,7].  It  would  require  many 
pages  to  summarize  all  of  the  previous  work  in  this  area.  For  the  purposes  of  this  paper, 
the  summary  will  detail  only  those  results  which  have  a  direct  bearing  on  the  simulation 
being  used  to  evaluate  the  two  reliability  growth  models. 

A  simulation  to  evaluate  these  models  was  constructed  by  Captain  James  Drake 
[Ref.  1:  pp.  36-46].  This  simulation  was  written  to  handle  growth  patterns  where  actual 
reliability  was  constant  or  growing  at  a  fixed  rate.  Since  one  of  the  objectives  of  this 
paper  is  to  expand  the  number  of  reliability  growth  patterns  that  can  be  simulated  it  is 
important,  at  this  time,  to  summarize  Captain  Drake's  work  to  include  the  assumptions 
necessary  to  run  the  simulation  and  some  of  the  limitations. 

Many  reliability  estimators  are  too  mathematically  complex  to  allow  closed  form 
solutions  of  their  properties.  In  order  to  compare  these  properties  computer  simulation 
is  one  of  the  only  techniques  available.  The  simulation  constructed  by  Captain  Drake 
builds  a  reliability  growth  pattern  and  generates  test  data  based  on  known  system  reli- 
abilities using  the  geometric  distribution.  The  discrete  reliability  growth  model  being 
evaluated  has  access  to  this  test  data  for  reliability  estimation.  The  test  data  generated 
by  the  computer  simulation  is  typically  the  number  of  trials  up  to  and  including  failure 
and  the  cause  of  system  failure  for  each  predesignated  failure  in  a  phase. 
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The  inputs  required  to  run  the  original  simulation  are  [Ref.  1:  pp.  40-41]: 

1.  Number  of  testing  phases. 

2.  Number  of  system  failures  allowed  in  each  phase. 

3.  Number  of  possible  failure  causes. 

4.  Probability  of  non-occurrence  of  a  particular  failure  cause  in  the  first  phase. 

5.  Reliability  growth  fraction. 

6.  Discounting  option  and  parameters. 

Several  assumptions  were  necessary  to  produce  an  adequate  simulation  based  on 
these  inputs.  These  assumptions  are  grouped  into  two  categories  -  reliability  growth 
pattern  assumptions  and  failure  cause  assumptions.  The  assumptions  required  for  simu- 
lating the  reliability  growth  pattern  are  [Ref.  1:  pp.  37-38]: 

1.  The  reliability  growth  pattern  is  non-decreasing. 

2.  System  reliability  changes  only  at  phase  boundaries. 

3.  Equipment  improvements  are  implemented  immediately  after  a  phase  ends  and 
before  any  further  testing. 

4.  Failure  causes  are  corrected  only  at  the  end  of  a  phase. 

5.  Each  design  improvement  or  repair  removes  a  fixed  fraction  of  the  probability  of 
reoccurrence  for  the  corresponding  failure  cause. 

These  assumptions  narrowed  the  scope  of  the  reliability  growth  patterns  that  could 
be  evaluated.  The  first  assumption  does  not  allow  the  model  to  track,  any  reliability 
growth  patterns  that  exhibit  periods  of  declining  reliability.  While  for  the  most  part  ac- 
tual system  reliabilities  will  be  non-decreasing,  this  restriction  does  preclude  a  complete 
evaluation  of  the  reliability  estimators  under  consideration.  Assumptions  two  and  three 
are  consistent  with  the  previously  stated  definition  of  a  testing  phase.  Assumption  three 
does  not  allow  modeling  of  long  term  design  changes  that  while  indicated,  may  be  post- 
poned for  whatever  reason.  Assumption  five  represents  the  method  that  is  used  in  the 
simulation  to  model  the  impact  of  design  changes  or  improvements.  According  to  this 
assumption,  any  improvement  or  design  change  will  remove  a  fixed  fraction  (user  input 
number  five)  of  the  probability  of  occurrence  for  a  failure  cause.  The  underlying  as- 
sumption is  that  all  improvements  are  equally  effective. 

These  assumptions  were  necessary  to  generate  the  type  of  reliability  growth  patterns 
desired  in  Captain  Drake's  thesis.  By  introducing  the  concept  of  fixed  phase  reliability 
and  modifying  the  original  simulation  accordingly,  four  of  these  five  assumptions  were 
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no  longer  necessary.  The  exact  methodology  used  to  effect  this  modification  will  be  de- 
tailed in  the  next  chapter. 

The  last  category  of  assumptions  in  the  original  simulation  are  the  failure  cause 
determination  assumptions.  These  assumptions  are  [Ref.    1  :  p.  39]: 

1.  There  exists  a  finite  number  of  possible  failure  causes. 

2.  Each  failure  cause  has  a  fixed  probability  of  occurrence  in  each  phase. 

3.  System  reliability  can  be  modeled  as  a  series  system  of  the  failure  causes. 

4.  Each  failure  cause  is  stochastically  independent  of  the  other  failure  causes. 

Assumption  three  dictates  the  method  by  which  the  actual  system  reliability  is 
computed  in  the  simulation.  Since  the  occurrence  of  a  failure  cause,  by  definition,  means 
system  failure,  modeling  the  system  as  a  series  is  acceptable.  This  implies  then  that  the 
actual  system  reliability  may  be  mathematically  expressed  as: 

n 

R  =  YYl-pd  (2.8) 

(=i 

In  the  above  equation  P,  is  the  probability  of  occurrence  of  the  ith  failure  cause  and 
n  is  the  total  number  of  failure  causes. 

The  simulation  is  able  to  generate  descriptive  statistics  about  each  of  the  reliability 
growth  models  being  examined.  These  statistics  include  the  mean,  its  confidence  interval 
and  the  standard  deviation  for  each  reliability  estimate  at  each  phase.  In  order  to 
produce  these  statistics  the  simulation  should  be  replicated  a  sufficient  number  of  times. 
The  mean  and  the  standard  deviation  are  computed  in  the  usual  manner.  The  confidence 
interval  calculation  assumes  a  normal  distribution.  For  the  purposes  of  this  paper,  500 
replications  of  each  set  of  input  data  were  conducted  so  the  normality  assumption 
should  be  reasonable. 

One  of  the  limitations  of  the  model  in  addition  to  the  restricted  number  of  reliability 
growth  patterns  that  could  be  analyzed  was  in  the  application  of  the  discounting  meth- 
ods. The  full  range  of  input  parameters  could  not  be  explored  due,  in  large  measure,  to 
computer  limitations.  This  limitation  typically  manifested  itself  at  relatively  high  system 
reliability.  If  a  failure  cause  with  a  very  low  probability  of  occurrence  caused  system 
failure  early  and  if  the  input  parameters  for  the  discounting  method  were  set  such  that 
a  relatively  large  fraction  of  this  failure  was  discounted  over  a  relatively  short  number 
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of  successful  trials  then  it  was  possible  for  this  early  failure  to  be  discounted  to  essen- 
tially zero.  When  the  adjusted  trials  were  calculated  utilizing  Equation  2.3 
(ADJUSTED  TRIALS  =      /^Jl^L  TRIALS         ^  ^  ^  ^^  approached  infm. 

ADJ Go  /  £,£/  r AlLU Kt, 
ity  creating  a  system  error  in  the  computer. 

This  problem  was  solved  by  specifying  a  lower  bound  for  the  adjusted  failure.  If  the 
failure  was  discounted  so  that  its  adjusted  value  was  less  than  .0000001  then  the  adjusted 
failure  was  set  to  .0000001.  This  modification  allowed  the  full  range  of  input  parameters 
for  the  discounting  methods  to  be  evaluated.  The  effect  on  the  estimate  of  system  reli- 
ability is  negligible. 
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III.      METHODOLOGY 

One  of  the  major  objectives  of  this  paper  is  to  expand  the  simulation  described  in 
the  previous  chapter  so  that  a  greater  variety  of  reliability  growth  patterns  could  be  used 
to  evaluate  the  two  reliability  growth  models.  This  was  accomplished  through  a  modifi- 
cation of  the  simulation  that  allowed  the  user  to  input  the  desired  system  reliabilities  at 
each  phase.  This  chapter  will  detail  the  method  that  was  used  to  effect  this  modification 
along  with  a  description  of  the  reliability  growth  patterns  and  failure  discounting  options 
that  were  used. 

A.      FIXED  PHASE  RELIABILITY 

One  of  the  limitations  of  the  original  simulation  was  that  only  constant  reliability 
or  reliability  that  increased  at  a  constant  rate  could  be  modeled.  If  a  predesignated  fail- 
ure cause  resulted  in  a  failure  then  that  failure  cause's  probability  of  occurrence  on  any 
succeeding  phase  was  reduced  by  a  predetermined  constant  amount.  Thus,  once  the  user 
of  the  simulation  established  the  initial  inputs  for  phase  one,  he  had  very  limited  control 
over  the  actual  system  reliabilities  in  the  subsequent  phases. 

In  order  to  evaluate  the  two  reliability  growth  models  under  a  greater  variety  of 
potential  reliability  growth  patterns  the  concept  of  fixed  phase  reliability  was  introduced 
as  an  option  to  the  original  simulation.  With  this  modification  of  the  program,  the  user 
has  the  ability  to  fix  the  actual  system  reliability  at  each  phase.  Thus,  the  reliability 
growth  models  can  be  used  to  estimate  system  reliability  under  any  reliability  growth 
pattern  the  user  desires. 

Incorporating  fixed  phase  reliability  into  the  simulation  necessitated  an  expansion 
of  the  user-provided  input.  The  basic  premise  used  in  computing  actual  system  reliability 
remains  the  same.  This  premise  is  that  system  reliability  is  the  product  of  the  probabili- 

n 

ties  of  success  of  the  failure  causes  (  Equation  2.8,  R  =  I~I(1  —  /*,)  )•  Originally,  the 
simulation  required  that  the  user  input  the  probability  of  success  for  each  failure  cause 
for  the  first  phase.  After  that,  the  fractional  improvement  factor  dictates  the  actual  sys- 
tem reliability  in  succeeding  phases.  If  the  option  of  fixing  the  phase  reliabilities  is  cho- 
sen, then  the  user  must  specify  the  probability  of  success  for  each  failure  cause  in  each 
phase.  Equation  2.8  is  then  applied  at  each  phase  to  yield  the  system  reliability. 
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The  table  below  illustrates  the  inputs  that  would  be  required  to  produce  a  growth 
pattern  that  initially  declines  and  then  increases  rapidly.  The  scenario  depicted  is  a  sim- 
ple one  involving  five  test  phases  with  one  failure  per  phase  and  two  failure  causes,  X 
and  Y.   Desired  reliability  is  user  determined. 

Table  5.      FIXED  PHASE  RELIABILITY  EXAMPLE 


PHASE 

DESIRED 
RELIABIL- 
ITY 

USER  INPUTS 

ACTUAL 

RELI- 
ABILITY 

PROBABILITY  OF  SUCCESS 

CAUSE  X 

CAUSE  Y 

I 

0.80 

0.90 

0.89 

0.801 

2 

0.70 

0.83 

0.84 

0.697 

3 

0.65 

0.77 

0.84 

0.647 

4 

0.75 

0.84 

0.89 

0.748 

5 

0.90 

0.94 

0.96 

0.902 

The  actual  reliability  for  each  phase  was  computed  using  Equation  2.8.  Consider 
phase  one,  for  example.  The  desired  system  reliability  is  0.80.  To  achieve  this  reliability, 
the  probabilities  of  success  for  failure  causes  X  and  Y  were  input  as  0.90  and  0.89  re- 
spectively. This  yielded  a  computed  system  reliability  of: 


R  = 


rj(l  -  p.)  =  .90  x  .89  -  0.801 


&*i 


Figure  1  is  a  graphical  representation  of  the  reliabilities  computed  in  the  table.  The 
original  simulation  could  not  have  produced  this  reliability  pattern  since  declining  reli- 
ability is  present.  This  scenario  may  occur  for  systems  during  a  portion  of  their  devel- 
opment. While  it  is  true  that,  in  the  long  run,  most  systems  exhibit  non-decreasing 
reliabilities  it  is  important  to  consider  decreasing  reliability  when  evaluating  growth 
models. 

In  order  to  invoke  this  ability  to  specify  reliability  at  each  phase,  the  number  of  user 
provided  inputs  increased  from  two  to  ten.  If  there  had  been  two  failures  allowed  in  each 
of  the  five  testing  phases  then  the  user  inputs  would  have  increased  from  ten  to  twenty. 
Therefore,  while  the  option  of  allowing  the  user  of  the  simulation  to  control  the  actual 
system  reliability  at  each  phase  significantly  increases  the  variety  of  different  reliability 
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Figure  1.      Fixed  Phase  Reliability  Example 

growth  patterns  that  can  be  analyzed,  a  rather  large  increase  in  the  volume  of  input  data 
is  required. 

B.      RELIABILITY  GROWTH  PATTERNS 

To  more  fully  evaluate  the  properties  and  trends  of  the  two  reliability  growth 
models  described  in  the  previous  chapter,  it  was  necessary  to  analyze  their  performance 
with  respect  to  a  variety  of  different  reliability  patterns.  The  eight  growth  patterns  that 
were  analyzed  in  this  paper  will  be  presented  in  this  section. 

In  conjunction  with  the  reliability  growth  patterns,  another  important  impact  on 
the  performance  of  the  reliability  estimators  was  the  choice  of  discounting  method.  The 
table  below  depicts  the  various  combinations  of  parameters  for  the  standard  discount 
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method  that  were  analyzed.  The  discount  fraction  is  the  amount  of  a  previous  failure 
that  is  removed  each  time  the  discount  method  is  applied  while  the  discount  interval  is 
the  number  of  successful  trials  between  applications.  The  Lloyd  discount  method  only 
has  one  parameter,  the  confidence  interval.  For  the  purposes  of  this  paper,  the  Lloyd 
method  was  applied  with  confidence  values  of  .8  and  .9. 

Table  6.      STANDARD  DISCOUNT  METHOD  COMBINATIONS 


STANDARD  DISCOUNT  METHOD 

DISCOUNT  FRACTION 

DISCOUNT  INTERVAL 

0.00 

— 

0.25 

1 

0.25 

3 

0.25 

6 

0.25 

15 

0.50 

3 

0.50 

6 

0.50 

15 

0.75 

3 

0.75 

6 

0.75 

15 

The  scenarios  used  to  analyze  the  reliability  growth  models  in  this  paper  involved 
ten  testing  phases,  i.e.,  testing  was  conducted  until  ten  changes  had  been  made.  A  total 
of  five  failure  causes  were  considered  in  each  test  and  a  limit  of  one  failure  per  phase 
was  established.  The  simulation  is  able  to  handle  any  number  of  test  phases,  any  num- 
ber of  failure  causes,  and  any  number  of  failures  per  phase.  The  only  limitation  is  the 
capacity  of  the  computer  on  which  the  simulation  is  run.  Another  practical  limitation 
involving  the  fixed  reliability  option  is  the  volume  of  input  data  that  would  accompany 
a  test  design  involving  numerous  phases  or  failure  causes.  As  can  be  seen  by  the  reli- 
ability growth  patterns  and  user  input  tables  following,  the  probabilities  of  success  for 
the  failure  causes  were  allowed  to  fluctuate.  The  only  requirement  was  that  the  reliability 
growth  pattern  established  accurately  reflected  the  desired  growth  pattern. 

Figure  2  and  Table  7  describe  the  first  reliability  growth  pattern.  This  is  not  a  very 
conventional  pattern  in  that  most  estimators  assume  concave  growth  patterns  of  the 
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form  shown  in  Figures  5  and  6.  However,  this  type  of  reliability  growth  pattern  would 
not  be  unusual  in  situations  where  the  exact  method  or  technology  required  to  correct 
a  failure-causing  defect  is  not  immediately  available  but,  as  the  system  evolves  and  per- 
sonnel become  more  familiar  with  it,  the  the  failure  correction  process  proceeds  more 
efficiently. 

Figure  3  depicts  a  decreasing  reliability  growth  pattern.  In  this  instance,  an  at- 
tempted improvement  actually  caused  system  reliability  to  decrease.  This  situation. 
which  is  not  effectively  addressed  by  any  conventional  reliability  estimators,  could  pos- 
sibly occur  in  experimental  or  technologically  advanced  systems  where  the  complete 
ramifications  of  design  changes  or  corrections  are  not  known. 

Figure  4  represents  a  scenario  in  which  the  system  under  consideration  attains  a 
moderately  high  reliability  and  then  stagnates  at  that  level  for  several  phases  before 
starting  to  improve  again.  This  scenario  could  occur  if  the  exact  cause  of  system  failure 
is  difficult  to  assess  and  would  not  be  uncommon  if  the  system  being  developed  is  highly 
complex. 

Figures  5  and  6  depict  conventional  reliability  growth  patterns  one  would  expect  to 
encounter  when  evaluating  the  majority  of  systems.  In  Figure  5,  the  system  reliability- 
increases  rapidly  to  0.99  and  then  remains  constant  whereas,  in  Figure  6,  the  system  re- 
liability increases  rapidly  to  0.90  before  becoming  constant.  It  is  important  in  the  eval- 
uation of  the  reliability  growth  models  to  compare  their  performance  in  relatively 
atypical,  although  not  uncommon,  scenarios  such  as  those  shown  in  Figures  2  through 
5  with  their  performance  in  more  conventional  situations. 

Figures  7,  8,  and  9  describe  a  constant  system  reliability  of  0.90,  0.60,  and  0.40  re- 
spectively. While  it  would  be  rare  to  find  these  system  reliabilities  occurring  in  actual 
practice,  their  inclusion  will  illuminate  some  of  the  trends  of  the  growth  models  under 
the  various  failure  discounting  methodologies. 

Each  reliability  growth  pattern  was  simulated  500  times  for  each  of  the  failure  dis- 
counting combinations  described  previously.  This  will  enable  a  more  accurate  appraisal 
of  the  capabilities  and  limitations  of  the  two  reliability  growth  models  being  considered. 
In  addition,  the  most  commonly  used  estimator  of  reliability,  the  single  phase  maximum 
likelihood  estimate,  will  be  included  on  all  figures  so  that  a  basis  for  comparing  the  two 
reliability  growth  estimators  exists.  The  single  phase  maximum  likelihood  estimate,  in 
contrast  to  the  maximum  likelihood  estimate  with  failure  discounting  being  evaluated  in 
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Table  7.      PATTERN  I  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

0 

£ 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.85 

.86 

.90 

.91 

.93 

.95 

.97 

.99 

.99 

.998 

.84 

.85 

.87 

.90 

.92 

.95 

.97 

.99 

.99 

.998 

J) 

.83 

.84 

.86 

.88 

.90 

.93 

.96 

.98 

.99 

.998 

4 

.83 

.84 

.85 

.87 

.89 

.92 

.94 

.975 

.99 

.998 

.5 

.SI 

.83 

.84 

.86 

.89 

.91 

.94 

.961 

.99 

.998 

~>~> 


PATTERN  II 
INCREASING  -  DECREASING  -  INCREASING 


10 


PHASE 


Figure  3.      Pattern  II 


Table  8. 

PATTERN  II  USER  INPUTS 

CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.98 

.98 

.09 

.09 

.99 

.99 

.09 

.90 

.90 

.99 

2 

.95 

.97 

.98 

.98 

.78 

.93 

.98 

.99 

.00 

.09 

3 

.82 

.93 

.96 

.72 

.90 

.95 

.9  7 

.98 

.08 

.OS 

4 

.80 

.92 

.96 

.96 

.96 

.96 

.96 

.97 

.08 

.98 

5 

.66 

.85 

.90 

.90 

.90 

.90 

.94 

.90 

.06 

.96 
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PATTERN  III 
INCREASING  -  CONSTANT  -  INCREASING 


Id 

e 


-  ^^— «  *  t 


10 


PHASE 


Figure  4.      Pattern  III 


Table  9.       PATTERN  III  USER  INPUTS 

CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.86 

.93 

.97 

.97 

.98 

.98 

.98 

.99 

.99 

.99 

2 

.86 

.93 

.97 

.97 

.98 

.98 

.98 

.99 

.99 

.99 

3 

.86 

.93 

.97 

.97 

.98 

.98 

.98 

.99 

.99 

.99 

4 

.86 

.93 

.97 

.97 

.98 

.98 

.98 

.99 

.99 

.99 

5 

.74 

.80 

.90 

.90 

.87 

.87 

.93 

.94 

.94 

.94 
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PATTERN  IV 
RAPID  INCREASE  TO  HIGH  RELIABILITY 


4  0 

PHASE 


10 


Figure  5.      Pattern  IV 


Table  10.      PATTERN  IV  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

-> 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.98 

.99 

.99 

.995 

.998 

.998 

.998 

.998 

.998 

.998 

2 

.95 

.98 

.99 

.995 

.998 

.998 

.998 

.998 

.998 

.998 

5 

.82 

.96 

.99 

.995 

.998 

.998 

.998 

.998 

.998 

.998 

4 

.80 

.96 

.99 

.995 

.998 

.998 

.998 

.998 

.998 

.998 

5 

.66 

.90 

.99 

.995 

.998 

.998 

.998 

.998 

.998 

.998 
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PATTERN  V 

RAPID  INCREASE  TO  MODERATELY  HIGH  RELIABILITY 

q 

DO 

o 

us 

>;    O 

UJ 

e  * 

o 

w 

o 

1 

i              i              i              i              i              i              i              i 

|. 

2                         4                         a                         a 

PHASE 

10 

Figure  6.      Pattern  V 


Table  11.       PATTERN  V  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.98 

.99 

.99 

.99 

.99 

.99 

.99 

.99 

.99 

.99 

2 

.95 

.98 

.99 

.99 

.99 

.99 

.99 

.99 

.99 

.99 

3 

.82 

.96 

.98 

.98 

.98 

.98 

.98 

.98 

.98 

.98 

4 

.80 

.96 

.97 

.975 

.975 

.975 

.975 

.975 

.975 

.975 

5 

.66 

.90 

.96 

.961 

.961 

.961 

.961 

.961 

.961 

.961 
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PATTERN  VI 

CONSTANT  MODERATELY  HIGH  RELIABILITY 

q 

d 

CO 

>;    d 

- 

I 

UJ 

d 

d 

1 

i 

i              i             i              i             i              i              i 

I 

2 

4sa 

PHASE 

10 

Figure  7.      Pattern  VI 


Table  12.      PATTERN  VI  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

I 

2 

3 

4 

5 

6 

i 
i 

8 

9 

10 

1 

.99 

.98 

.98 

.975 

.97 

.97 

.975 

.98 

.98 

.94 

2 

.98 

.98 

.975 

.97 

.99 

.99 

.97 

.975 

.98 

.98 

i 

j 

.98 

.975 

.97 

.99 

.98 

.98 

.99 

.97 

.975 

.(>8 

4 

.975 

.97 

.99 

.98 

.98 

.98 

.98 

.99 
~ 98 

.97 

.975 

5 

.97 

.99 

.98 

.98 

.975 

.975 

.98 

.99 

.4  7 

27 


PATTERN  VII 

CONSTANT  MODERATE  RELIABILITY 

q 

09 

6 

- 

£  ^ 

s 

Lli 
EC     *, 

a 

ri 

I 

i 

i              i              i              i              i              i 

i              j_ 

2 
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PHASE 

10 

Figure  8.      Pattern  VII 


Table  13.      PATTERN  VII  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.99 

.78 

.90 

.96 

.90 

.90 

.96 

.90 

.78 

.99 

2 

.78 

.90 

.96 

.90 

.99 

.99 

.90 

.96 

.90 

.78 

3 

.90 

.96 

.90 

.99 

.78 

.78 

.99 

.90 

.96 

.90 

4 

.96 

.90 

.99 

.78 

.90 

.90 

.78 

.99 

.90 

.96 

5 

.90 

.99 

.78 

.90 

.96 

.96 

.90 

.78 

.99 

.90 
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PATTERN  VIII 

CONSTANT  LOW  RELIABILITY 

- 

q 

to 
o 

to 

- 

LU 
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l 
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10 

Figure  9. 


Pattern  VIII 


Table  14.      PATTERN  VIII  USER  INPUTS 


CAUSE 

PHASE  PROBABILITY  OF  SUCCESS 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 
i 

.98 

.95 

.82 

.80 

.66 

.06 

.80 

.82 

.95 

.98 

L 

.95 

.82 

.80 

.66 

.98 

.98 

.66 

.80 

.82 

.95 

.82 

.SO 

.66 

.98 

.95 

.95 

.98 

.66 

.80 

.82 

4 

.80 

.66 

.98 

.95 

.82 

.82 

.95 

.98 

.60 

.80 

5 

.66 

.98 

.95 

.82 

.80 

.80 

.82 

.95 

.l»8 

.66 
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this  paper,  only  considers  the  number  of  trials  and  failures  in  the  current  phase.  All 
previous  data  is  discarded  since,  presumably,  the  system  reliability  fluctuates  from  phase 
to  phase  thereby  negating  the  underlying  assumption  of  constant  reliability. 

Of  the  eight  reliability  growth  patterns  being  simulated,  only  four  (including  the 
three  cases  of  constant  system  reliability,  Figures  7,  8,  and  9)  could  have  been  produced 
with  the  simulation  as  originally  written.  Therefore,  the  addition  of  fixed  phase  reliability 
will  enable  a  more  rigorous  examination  of  the  proposed  reliability  estimators. 
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IV.   EVALUATION  AND  INTERPRETATION 

This  chapter  describes  results  of  the  simulation  runs  for  each  pattern.  The  per- 
formance of  the  models  are  reduced  to  graphical  form  to  facilitate  comparison.  Only  a 
few  of  the  more  representative  runs  are  included  in  this  chapter  due  to  time  and  space 
constraints.  The  graphical  results  of  many  of  the  remaining  simulation  runs  are  in  Ap- 
pendix C  . 

The  analysis  is  organized  according  to  the  reliability  growth  patterns  presented  in 
the  previous  chapter.  The  performance  of  each  of  the  two  reliability  growth  models  will 
be  evaluated  with  respect  to  the  growth  pattern.  The  analysis  focuses  on  how  well  the 
growtli  model  tracked  the  actual  reliability  growth  pattern.  Included  in  this  evaluation 
is  an  examination  of  the  standard  deviations  of  the  two  estimators. 

It  is  not  the  purpose  of  this  research  to  conduct  enough  simulation  runs  so  that 
precise  rules  can  be  established  for  the  selection  of  discounting  parameters.  This  would 
require  analyzing  many  more  different  types  of  reliability  growth  patterns.  In  addition, 
the  choice  of  discounting  methods  and  their  respective  parameters  is  more  a  function  of 
the  particular  system  being  evaluated  than  a  function  of  the  reliability  growth  model. 
Each  system  will  have  varying  characteristics  as  will  the  personnel  responsible  for  con- 
ducting the  testing  procedures. 

The  reliability  growth  patterns  will  be  referred  to  by  their  respective  Roman  nu- 
meral designations.  These  numerals  are  listed  on  the  figures  describing  each  pattern  in 
the  previous  chapter  and  are  summarized  below  for  completeness: 

1.  Pattern  I  -  Convexly  increasing  reliability. 

2.  Pattern  II  -  Reliability  initially  increasing  then  decreases  for  several  phases  before 
resuming  its  upward  trend. 

3.  Pattern  III  -  Reliability  increases  rapidly  to  approximately  0.80.  Reliability  then 
remains  constant  for  several  phases  before  increasing  to  its  final  level  of  approxi- 
mately 0.90. 

4.  Pattern  IV  -  Reliability  increases  rapidly  to  approximately  0.99. 

5.  Pattern  V  -  Reliability  increases  rapidly  to  approximately  0.90. 

6.  Pattern  VI  -  Pliability  starts  and  remains  constant  at  0.90. 

7.  Pattern  VII  -  Reliability  starts  and  remains  constant  at  0.60. 

8.  Pattern  VIII  -  Reliabilitv  starts  and  remains  constant  at  0.40. 
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A.      CONSTANT  RELIABILITIES  --  PATTERNS  VI,  VII,  AND  VIII 

1.      Standard  Discount  Method 


PATTERN  VI  -  NO  DISCOUNTING 


o* 


— « ACRJAL  RELIABILITY 

— 0 —  MLE  SINGLE  PHASE 

— V —  EXPONENTIAL  REGRESSION 

— + —  MLEFD 


10 


PHASE 


Figure   10.      Pattern  VI-  No  Discounting 

Figure  10  shows  the  performance  of  the  two  reliability  growth  models  with  re- 
spect to  Pattern  VI.  In  this  particular  figure  the  discount  fraction,  F,  is  set  at  zero  (pre- 
vious system  failures  are  not  discounted).  As  one  might  expect,  both  models  performed 
well.  Since  the  actual  system  reliability  is  constant  across  all  phases,  the  basic  underlying 
assumption  of  constant  reliability  for  the  maximum  likelihood  estimate  is  valid.  There- 
fore, previous  data  is  compatible  with  current  data  without  the  use  of  failure  discounting 
techniques.  Thus,  including  all  previous  failure  data  such  as  done  in  the  Maximum 
Likelihood  Estimate  with  Failure  Discounting  (MLEFD)  will  lead  to  a  more  accurate 
estimation  of  reliability.  It  is  also  interesting  to  note  that  the  single  phase  maximum 
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likelihood  estimate  discussed  briefly  in  the  previous  chapter  resulted  in  an  estimate  of 
0.743  for  phase  ten.  The  single  phase  maximum  likelihood  estimate  considers  only  the 
number  of  trials  and  failures  in  the  current  phase  and  ignores  past  test  data.  The  ex- 
pected value  of  this  estimate,  using  Equation  2.5  is  0.744.  thus  supporting  the  validity 
of  the  simulation  process. 


PATTERN  VI  -  F  =  .25.  I  =  6 


r— 


— ♦ —  ACTUAL  RELIABILITY 
— □ —  MLE  SINGLE  PHASE 

— 7 —  EXPONENTIAL  REGRESSION 
— + —  MLEFD 


10 


PHASE 


Figure  11. 


Pattern  VI,  F  =  .25  and  I 


Both  models  did  very  well  in  terms  of  the  stability  of  the  estimate  when  the 
actual  system  reliability  was  high.  This  is  largely  due  to  the  fact  that  high  system  reli- 
ability implies  that  more  test  trials  will  be  required  to  produce  a  failure.  Mote  test  trials 
translates  into  additional  data  for  the  two  models  and,  therefore,  a  less  variable  estimate. 
It  should  be  noted,  however,  that  when  actual  system  reliability  is  constant,  the  MLEFD 
does   better  than   the   exponential  regression  model.   The  standard  deviation  of  the 
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MLEFD  is  less  than  half  that  of  the  exponential  regression  estimate.  Appendix  C  also 
contains  the  graphs  of  the  standard  deviations  of  the  two  estimates  for  all  eight  patterns 
for  the  interested  reader. 


PATTERN  VIII  -  LLOYD,  CI  =  .9 


-« —  ACTUAL  RELIABILITY 
-D —  MLE  SINGLE  PHASE 

-7 —  EXPONENTIAL  REGRESSION 
-+ —  MLEFD 


5 


10 


PHASE 


Figure  12.      Pattern  VIII,  Lloyd  method  -  CI  =  .9 

The  standard  discount  method  will  lead  to  an  overestimation  of  reliability  when 
applied  to  the  MLEFD  for  the  constant  system  reliability  patterns.  Figure  11  depicts 
the  results  of  applying  this  method  with  the  parameter  values  set  at  F  =  0.25  and  I  = 
6.  While  only  slightly  affecting  the  exponential  regression  model,  discounting  previous 
failures  in  this  manner  resulted  in  the  MLEFD  overestimating  system  reliability  fairly 
significantly  after  phase  four.  This  is  not  unexpected  since,  as  stated  before,  the  as- 
sumption of  constant  system  reliability  is  valid  in  this  instance.  It  is  important  to  note, 
however,  the  sensitivity  of  the  MLEFD  model  to  discount  procedures  compared  to  the 
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exponential  regression  model.  Despite  using  a  relatively  small  value  for  F  the  predicted 
reliabilities  were  markedly  altered.  This  characteristic  of  the  MLEFD  is  also  evident  in 
the  patterns  to  follow. 

2.      Lloyd  Discount  Method 

The  Lloyd  discount  failure  discounting  method  has  only  one  parameter,  the 
confidence  interval.  For  the  purposes  of  this  study  the  only  values  for  CI  that  were  ad- 
dressed were  0.80  and  0.90.  The  formula  for  applying  the  Lloyd  discount  method. 
Equation  2.2,  is  constructed  so  that  the  higher  the  value  of  CI,  the  less  a  previous  failure 
is  discounted.  Figure  12  graphically  represents  Pattern  VIII  and  the  results  of  applying 
the  Lloyd  method  with  CI  =  0.90  to  the  two  models.  Once  again,  the  effect  on  the 
MLEFD  is  rather  dramatic  with  a  phase  ten  estimate  of  reliability  of  approximately  0.70 
while  the  actual  system  reliability  is  0.40.  The  effect  on  the  exponential  regression  model, 
although  significant  after  phase  seven,  is  not  nearly  so  severe.  This  marked  effect  of 
applying  the  Lloyd  discount  method  to  the  two  reliability  growth  models  continued  with 
the  other  patterns  as  will  be  seen.  Again,  the  important  thing  to  note  is  the  extreme 
sensitivity  of  the  MLEFD  to  failure  discounting  in  general  and  to  the  chosen  values  of 
the  discount  parameters  in  particular. 

B.      RAPID  RELIABILITY  GROWTH  --  PATTERNS  IV  AND  V 

The  discussion  of  the  results  achieved  for  these  patterns  will  center  around  Pattern 
V.  Although  the  tendencies  of  the  two  models  were  the  same  for  both  patterns,  the 
graphical  portrayal  of  identified  trends  is  better  observed  with  this  pattern  than  with 
Pattern  IV  where  the  final  system  reliability  approaches  0.99. 

1.      Standard  Discount  Method 

Figure  13  depicts  Pattern  V  with  the  discount  fraction,  F,  set  at  zero.  Both  the 
MLEFD  and  the  exponential  regression  model  perform  well.  It  is  significant  to  note, 
however,  that  the  exponential  regression  model  converged  to  the  actual  system  reliability 
much  faster  than  did  the  MLEFD.  This  is  due  in  large  part  to  the  use  of  linear  regression 
techniques  in  this  model.  The  standard  deviation  of  the  exponential  regression  was 
comparable  to  the  MLEFD  for  this  pattern  although  still  slightly  larger.  Both  the 
MLEFD  and  the  exponential  regression  model  outperformed  the  single  phase  maximum 
likelihood  estimate  thus  lending  validity  to  the  use  of  reliability  growth  models  to  esti- 
mate system  reliability. 

Figure  14  shows  Pattern  V  with  F  =  0.50  and  I  =  3  trials.  This  high  a  rate  of 
failure  discounting  caused  the  MLEFD  to  yield  a  phase  ten  estimate  of  0.997  when  the 
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PATTERN  V  -  NO  DISCOUNTING 


PHASE 


Figure   13.      Pattern  V,  No  Discounting 

actual  reliability  was  0.900.  A  correlate  of  this  high  estimate  of  reliability  is  that  the 
standard  deviation  associated  with  the  MLEFD  became  very  small  relatively  quickly. 
The  sensitivity  of  the  MLEFD  to  failure  discounting  is  again  apparent  in  this  figure. 

The  exponential  regression  model,  on  the  other  hand,  does  remarkably  well.  It 
quickly  approached  actual  system  reliability.  However,  the  standard  deviation  associated 
with  this  estimate  remained  high  at  a  relatively  constant  value  of  0.20.  So,  while  the 
mean  estimate  of  system  reliability  was  very  good,  the  associated  variance  of  the  esti- 
mate is  greater  than  one  would  desire. 

The  MLEFD  performed  quite  well  when  F  was  set  at  0.25  and  I  was  put  at  15 
as  shown  in  Figure  15.  The  exponential  regression  model  did  slightly  overestimate  the 
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PATTERN  V  -  F  =  .50,  I  =  2 


— + —  ACTUAL  RELIABILITY 
— 0 —  MLE  SINGLE  PHASE 
—  7 —  EXPONENTIAL  REGRESSION 
— + —  MLEFD 
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10 


PHASE 


Figure   14.      Pattern  V,  F  =  .5  and  1  =  3 

system  reliability  after  the  sixth  phase  however,  it  was  more  proficient  at  matching  the 
actual  growth  of  reliability  in  the  system.  There  was  not  a  significant  difference  in  the 
standard  deviations  of  the  two  estimates. 
2.      Lloyd  Discount  Method. 

Figure  16  depicts  Pattern  V  with  a  confidence  interval  value  of  0.80.  Note  that, 
as  on  the  constant  system  reliability  patterns  discussed  previously,  application  of  the 
Lloyd  discount  method  in  its  derived  form  caused  both  estimators  to  predict  system  re- 
liabilities that  were  markedly  greater  than  the  actual  system  reliability.  This  trend  was 
consistent  no  matter  what  values  of  CI  were  used. 
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PATTERN  V  -  F  =  .25,  1  =  15 


♦ —  ACTUAL  RELIABIirTY 
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—  7 —  EXPONENTIAL  REGRESSION 

— + —  MLEFD 
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Figure   15.      Pattern  V,  F  =  .25  and  I  =   15 

The  conventional  estimate  of  reliability,  the  single  phase  maximum  likelihood 
estimate,  is  not  affected  by  discount  procedures  since  only  current  test  data  is  consid- 
ered. The  discounting  methods  addressed  in  this  paper  are  only  applied  to  failures  oc- 
curring in  previous  phases. 

C.      DECREASING  RELIABILITY  -  PATTERN  II 
1.      Standard  Discount  Method 

Figure  17  shows  Pattern  II  with  no  failure  discounting  applied.  Of  special  in- 
terest in  this  graph  is  the  performance  of  the  two  reliability  growth  models  during  the 
period  of  declining  reliability.  The  exponential  regression  model  was  the  only  model  that 
actually  demonstrated  a  decrease  in  reliability  although  it  still  overestimated  the  actual 
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PATTERN  V  -  LLOYD,  CI  -  .8 
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—  7 —  EXPONENTIAL  REGRESSION 
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Figure   16.      Pattern  V,  Lloyd  method,  CI  =  .8 

reliability  during  this  period.  The  MLEFD,  although  slowing  in  terms  of  growth  rate, 
failed  to  capture  the  trend  of  the  actual  reliability.  The  lack  of  ability  to  successfully 
predict  the  downward  trend  in  actual  reliability  on  the  part  of  the  MLFFD  was  present 
regardless  of  what  values  were  assigned  the  discount  parameters.  In  fact,  as  one  might 
expect,  discounting  previous  failures  exacerbated  the  problem. 

Figure  18  shows  Pattern  II  with  F  =  0.75  and  I  =  6.  The  exponential  re- 
gression model  did  very  well,  particularly  during  the  phases  where  system  reliability  was 
declining.  The  MLEFD,  in  contrast,  performed  poorly  overestimating  system  reliability 
significantly  after  phase  three  and  failing  to  indicate  that  system  reliability  decreased. 
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PATTERN  II  -  NO  DISCOUNTING 
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Figure  17.      Pattern  II,  No  Discounting 

The  standard  deviation  of  the  exponential  regression  estimate  still  exceeded  that 
of  the  MLEFD.  This  characteristic  is  fairly  consistent  for  all  the  patterns  that  were  ex- 
amined in  the  course  of  evaluating  these  two  models.  It  should  be  noted,  however,  that 
a  small  standard  deviation  does  little  good  if  the  estimator  is  yielding  markedly  incorrect 
predictions. 

2.      Lloyd  Discount  Method 

Figure  19  shows  Pattern  II  with  CI  =  0.90.  The  application  of  the  Lloyd 
method  profoundly  affected  the  performance  of  the  MLEFD  while  only  slightly  altering 
the  exponential  regression  model.  This  method  of  failure  discounting  did  cause  both 
models  to  overestimate  system  reliability  by  phase  nine  but,  again,  had  less  of  an  effect 
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PATTERN  II  -  F  =  .75.  I  =  5 


PHASE 


Figure   18.      Pattern  II,  F  =  .75,  I  =  6 

on  the  exponential  regression  model.  It  is  especially  noteworthy  to  observe  the  effect  of 
the  Lloyd  method  on  the  MLEFD  during  the  period  of  decreasing  system  reliability.  The 
real  sensitivity  of  the  MLEFD  to  discount  methods  and  parameters  is  particularly  evi- 
dent in  this  pattern. 

D.      INTERMITTANT  RELIABILITY  GROWTH  -  PATTERN  III 

1.      Standard  Discount  Method 

Figure  20  depicts  the  performance  of  the  two  models  when  no  failure  dis- 
counting is  applied.  The  exponential  regression  model  outperformed  the  MLEFD  at  ev- 
en' phase.  Again,  the  exponential  regression  model  succeeded  in  capturing  the  trend  of 
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PATTERN  II  -  LLOYD.  CI  =  .9 
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Figure   19.      Pattern  II,  Lloyd  method,  CI  =  .9 

the  reliability  growth  pattern  from  phase  four  to  phase  sj\  whereas  the  MLEFD  exhib- 
ited a  constant  rate  of  increase. 

When  the  discount  parameters  F  and  I  were  set  at  0.25  and  six  respectively,  the 
MLEFD  did  much  better  as  can  be  seen  in  Figure  21.  Both  models  converged  to  the 
correct  prediction  of  system  reliability  by  phase  ten.  However,  the  exponential  regression 
model  converged  to  the  correct  value  much  more  quickly  than  did  the  MLEFD.  The 
standard  deviations  of  the  two  estimates  were  not  too  significantly  different  from  each 
other  for  this  particular  pattern  and  discount  method. 

The  performance  of  the  two  models  on  reliability  growth  patterns  II  and  III 
demonstrate  one  of  the  inherent  advantages  of  the  exponential  regression  methodology. 
Since  this  method  makes  use  of  the  techniques  of  linear  regression,  it  is  better  able  to 
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PATTERN  III  -  NO  DISCOUNTING 
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Figure  20.      Pattern  III,  No  Discounting 

track  changing  system  reliability  than  is  the  MLEFD.  This  attribute  of  the  exponential 
regression  model  will  be  developed  further  in  the  next  chapter. 
2.      Lloyd  Discount  Method 

Figure  22  shows  how  the  models  performed  using  Lloyd  failure  discounting 
with  CI  =  0.90.  Again,  applying  the  Lloyd  method  caused  the  MLEFD  to  overestimate 
system  reliability  after  phase  four.  Some  slight  overestimation  did  occur  in  the  expo- 
nential regression  model  during  phases  nine  and  ten  however,  the  overall  elfect  was  to 
enable  the  model  to  better  track  actual  system  reliability  during  the  middle  phases. 
During  the  discussion  of  the  discounting  methodologies  in  Chapter  II,  it  was  noted  that 
the  Lloyd  failure  discount  method  is  applied  after  every  trial  that  did  not  result  in  the 
reoccurrence  of  a  previous  failure  cause.  This  means  of  application  is  a  likely  source  of 
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PATTERN  III  -  F  -  .25,  I  =  6 


PHASE 


Figure  21.      Pattern  III,  F  =  .25,  I  =  6 

the  overestimation  characteristics  of  both  models  when  Lloyd  discounting  is  applied. 
An  idea  on  a  possible  remedy  to  this  problem  will  be  presented  at  the  end  of  this  chapter. 

E.      CONVEX  RELIABILITY  GROWTH  -  PATTERN  I 

Most  reliability  growth  patterns  are  thought  to  be  essentially  concave  in  shape, 
much  like  Patterns  IV  and  V.  This  is  normally  a  valid  assumption  in  that  major  im- 
provements in  the  reliability  of  a  new  system  most  often  occur  during  the  early  stages 
of  development  as  the  obvious  flaws  in  the  design  and  manufacture  become  apparent. 
It  is  not  inconceivable,  however,  to  experience  a  period  of  convex  growth  such  as  shown 
in  Pattern  I.  One  would  want  to  have  confidence  in  the  chosen  reliability  estimator  to 
correctly  estimate  system  reliability  regardless  of  its  actual  form. 
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PATTERN  III  -  LLOYD,  CI  -  .9 
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Figure  22.      Pattern  III,  Lloyd  method,  CI  =  .9 

1.      Standard  Discount  Method 

Figure  23  shows  the  performance  of  the  two  models  with  no  failure  discounting 
being  applied.  It  is  apparent  that,  even  without  the  benefit  of  failure  discounting  tech- 
niques, the  exponential  regression  model  does  well.  It  is  able  to  reflect  the  trend  of  actual 
system  reliability  growth  and  the  actual  estimates  are  fairly  close  to  the  true  system  re- 
liability. The  MLEFD  does  not  do  so  well  and,  in  fact,  is  even  outperformed  by  the 
single  phase  maximum  likelihood  estimate  in  the  later  phases.  One  would  expect  in  this 
case  to  have  significant  results  when  failure  discounting  is  applied  due  to  the  demon- 
strated sensitivity  of  the  MLEFD  to  this  procedure. 
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PATTERN  I  -  NO  DISCOUNTING 


PHASE 


Figure  23.      Pattern  I,  No  Discounting 

Figure  24  depicts  Pattern  I  with  F  =  .75  and  I  =  6.  Although  failing  to  capture 
the  shape  of  the  actual  reliability  pattern,  the  phase  estimates  produced  by  the  MLEFD 
are  quite  good.  Fractionally  removing  75  percent  of  a  previous  failure  after  six  successful 
trials  is  a  substantial  discount  and  one  would  expect  that  such  a  large  discounting 
scheme  would  lend  itself  more  to  the  conventional  concave  reliability  growth  patterns. 
This  particular  type  of  scenario  would  benefit  greatly  by  the  ability  to  alter  the  discount 
parameters  during  the  course  of  the  test.  Initially,  one  would  not  want  to  discount  pre- 
vious failures  by  a  large  amount  since  actual  system  reliability  is  only  improving  mar- 
ginally. As  the  system  evolves  and  reliability  begins  to  make  substantial  jumps  then 
previous  failures  should  be  discounted  more.  Of  course,  this  presumes  a  knowledge  of 
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PAITERN  I  -  F  =  .75,  I 
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Figure  24.      Pattern  I,  F  =  .75  and  I  =  6 

the  actual  system  reliability  in  order  to  properly  design  such  a  plan  and  this  is  never  the 
case.  However,  in  depth  knowledge  of  the  proposed  system  could  indicate  the  expected 
shape  of  the  reliability  pattern  and  this  could  be  made  use  of  in  test  design.  Although 
changing  the  discount  parameters  during  the  course  of  the  test  is  possible  in  actual 
practice  it  is  not.  unfortunately,  possible  to  do  with  the  simulation  in  its  current  form. 
The  parameters  for  the  discount  procedures  remain  constant  once  they  are  input  at  the 
beginning  of  the  simulation. 

2.      Lloyd  Discount  Method 

Figure  25  shows  the  Lloyd  method  with  CI  =  0.90.  The  exponential  regression 
model  maintains  the  shape  of  the  true  system  reliability  while  converging  to  the  actual 
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PATTERN  I  -  LLOYD.  CI  -  .9 


— * —  ACTUAL  RELIABILITY 
— 0 —  MLE  SINGLE  PHASE 

—  7 —  EXPONENTIAL  REGRESSION 

—  + —  MLEFD 


10 


PHASE 


Figure  25.      Pattern  I,  Lloyd  method,  CI  =  .9 

reliability  by  phase  eight.  The  MLEFD  becomes  concave  in  shape  much  as  it  did  for  the 
standard  discount  method  shown  in  Figure  24.  However,  the  intermediate  phase  reli- 
ability estimates  are  very  good.  The  standard  deviation  of  the  MLEFD  continues  to  be 
less  than  that  of  the  exponential  regression  model  although,  as  shown  in  Chapter  II,  the 
mean  square  error  for  the  MLEFD  exceeds  that  of  the  exponential  regression  model. 

The  Lloyd  method  does  better  with  this  pattern  than  with  any  other.  Previ- 
ously, application  of  this  method  caused  both  models  to  overestimate  system  reliability 
quite  significantly.  In  this  case,  using  this  method  improved  the  performance  of  the  ex- 
ponential regression  model  and,  although  distorting  the  shape,  resulted  in  improved  re- 
liability estimates  from  the  MLEFD  as  well. 
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F.      LLOYD  FAILURE  DISCOUNTING  (  REVISITED  ) 

Applying  the  Lloyd  failure  discounting  method  in  its  current  form  consistently 
caused  both  models  to  overestimate  the  true  system  reliability.  This  was  true  for  the 
fixed  phase  reliability  growth  patterns  that  were  addressed  in  this  paper  as  well  as  Cor  the 
growth  patterns  that  were  analyzed  in  Captain  Drake's  work  previous  to  this  [P.cf.  1:  pp. 
64-65].  One  potential  source  of  this  problem  is  that  the  Lloyd  method  is  applied  after 
every  successful  trial. 

The  formula  for  computing  adjusted  failures  using  the  Lloyd  method  is  given  by 
Equation  2.2  repeated  below: 

ADJUSTED  FAILURE  =  1  -  (1  -  CI)~T      i/T>  0.  (2.2) 

If  T  equals  zero  then  the  adjusted  failure  value  is  set  to  one.  This  formula  is  used  to 
compute  adjusted  failures  after  each  trial  in  which  the  cause  for  a  previous  failure  did 
not  reoccur.  Since  the  problem  appears  to  lie  in  the  number  of  times  the  method  is  ap- 
plied the  inclusion  of  an  interval  may  lead  to  more  accurate  results. 

A  modified  version  of  the  Lloyd  discount  method  which  employs  an  interval  of 
application  is  as  follows: 

ADJUSTED  FAILURE  =  1  -  (1  -  CI)Tf      if  M  >  0  (4.1) 

where,  in  this  case, 

LDI  is  defined  as  the  Lloyd  Discount  Interval  and  becomes  another  parameter  that  can 
be  altered  by  the  user  of  the  simulation.  If  M  equals  zero  then  the  adjusted  failure  value 
is  set  at  one  as  before.  By  its  construction,  one  can  observe  that  for  all  values  of  T  less 
than  the  predesignated  LDI,  then  the  value  of  M  will  be  zero.  M  will  increase  by  one  for 
each  group  of  successful  trials  meeting  the  LDI.  Note  also  that  specifying  an  LDI  of  one 
allows  the  user  of  the  simulation  to  employ  the  Lloyd  method  in  its  original  form. 

Some  preliminary  runs  of  the  simulation  were  made  with  this  modification  included. 
The  results  were  encouraging.  Figure  12  examined  previously  illustrated  the  elfcct  of  the 
Lloyd  method  on  the  two  models  when  system  reliability  was  actually  a  constant  0.40. 
Figure  26  shows  the  result  of  applying  this  modified  version  of  the  Lloyd  method  with 
the  value  for  CI  remaining  at  0.90  and  the  LDI  set  at  eight.  Both  the  MLEFD  and  the 
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PATTERN  VIII  -  LLOYD,  CI  =  .9.  LDI  =  8 


-* —  ACTUAL  RELIABILITY 
-0 —  MLE  SINGLE  PHASE 

-7 —  EXPONENTIAL  REGRESSION 
-+ —  MLEFD 


i 
10 


PHASE 


Figure  26.      Pattern  VIII,  Modified  Lloyd  Method,  CI  =  .9,  LDI  =  3 

exponential  regression  model  performed  much  better  than  before.  The  LDI  value  of 
eight  resulted  in  similar  improvements  for  the  other  constant  system  reliability  patterns. 
The  results  on  the  other  patterns  were  similar  to  those  represented  in  Figure  17  but 
the  value  for  the  LDI  had  to  be  adjusted  for  each  pattern.  Before  any  specific  conclu- 
sions can  be  drawn  about  the  benefits  of  this  modification  many  more  computer  runs 
and  more  in  depth  analysis  is  required.  However,  based  on  these  preliminary  results,  the 
inclusion  of  an  interval  does  appear  to  have  some  merit. 


50 


V.      SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 

A.      SUMMARY 

The  purpose  of  this  paper  was  to  evaluate  two  discrete  reliability  growth  models. 
Each  of  these  models  has  the  ability  to  use  fractionally  discounted  failure  data  to  make 
an  estimate  of  the  actual  system  reliability  at  any  point  during  the  conduct  of  a  test. 
Fractionally  discounting  failures  occurring  early  in  a  test  procedure  allows  the  data  ac- 
cumulated during  the  initial  stages  of  a  test  to  be  used  in  current  reliability  estimates. 
Therefore,  if  the  discounting  procedure  is  performed  wisely,  more  accurate  reliability 
estimates  with  a  higher  degree  of  confidence  should  result. 

Two  procedures  for  discounting  previous  system  failures  were  described  in  Chapter 
II.  The  standard  discount  method  requires  the  user  to  specify  two  parameters,  the  frac- 
tion of  failure  to  be  removed  and  the  discount  interval.  The  fraction  of  failure  to  be  re- 
moved, referred  to  as  the  discount  fraction  or  F,  specifies  how  much  a  previous  failure 
is  reduced  upon  each  application  of  the  discount  procedure.  The  discount  interval,  I.  is 
simply  the  number  of  successful  trials  (trials  without  a  reoccurrence  of  the  particular 
failure  cause)  that  must  occur  between  applications  of  the  procedure.  The  other  failure 
discounting  method  was  proposed  by  David  Lloyd  and  is  referred  to  as  the  Lloyd  dis- 
count method.  This  method  utilizes  only  one  input  parameter,  the  level  of  confidence 
for  an  upper  confidence  interval  on  reliability.  This  method,  in  its  original  form,  is  ap- 
plied after  every  successful  trial.  A  modified  Lloyd  discounting  method  which  employs 
an  interval  of  application  was  derived  in  Chapter  IV. 

Two  discrete  reliability  growth  models  were  described.  The  first  model  is  a  form  of 
the  maximum  likelihood  estimate  with  the  addition  of  failure  discounting.  Since  the 
maximum  likelihood  estimate  assumes  a  constant  reliability  between  phases,  the  addition 
of  failure  discounting  will  enable  this  model  to  estimate  changing  system  reliability.  The 
Maximum  Likelihood  Estimate  with  Failure  Discounting,  or  MLEFD,  estimates  system 
reliability  as  the  ratio  of  the  number  of  successful  trials  to  the  total  number  of  trials  with 
this  ratio  being  computed  after  application  of  failure  discounting. 

The  other  model  was  developed  by  H.  Chernoff  and  W.YI.  Woods  and  is  a  deriva- 
tion of  an  exponential  single  phase  reliability  estimate.  This  model  employs  linear  re- 
gression techniques  to  estimate  reliability  after  any  change  has  been  made  to  the  unit 
of  hardware  under  test.  Because  of  this,  the  exponential  regression  model  is  able  to  track 
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changing  system  reliability  without  the  benefit  of  failure  discounting  although  the  use 
of  failure  discounting  can  enhance  the  model's  performance. 

The  means  of  evaluating  these  two  models  was  a  Monte  Carlo  simulation.  The 
simulation  used  was  originally  written  by  Captain  James  Drake  and  was  modified  so  that 
fixed  phase  reliabilities  could  be  modeled.  The  fixed  phase  reliability  modification  allows 
the  user  of  the  simulation  to  evaluate  the  performance  of  the  two  reliability  growth 
models  on  any  pattern  of  actual  reliability  growth  desired.  Thus,  periods  of  declining 
reliability,  periods  of  no  reliability  growth,  and  convexly  growing  reliability  among  oth- 
ers can  be  easily  modeled  and  the  performance  of  the  two  growth  models  analyzed. 

Eight  varying  reliability  growth  patterns  were  analyzed.  Both  of  the  reliability 
growth  models  were  evaluated  against  these  eight  patterns  and  under  13  different  failure 
discounting  combinations  including  no  failure  discounting.  Each  model,  pattern  and 
failure  discounting  combination  was  replicated  500  times.  The  ability  of  each  model  to 
accurately  track  the  actual  system  reliability  was  evaluated  along  with  an  examination 
of  the  stability  of  the  estimate  produced  by  the  model. 

B.      CONCLUSIONS 

The  evaluation  of  the  two  reliability  growth  models  on  the  eight  different  reliability 
growth  patterns  leads  to  several  general  conclusions.  The  most  obvious  conclusion  is 
that  the  use  of  either  one  of  these  growth  models  is  superior  to  the  standard  single  phase 
maximum  likelihood  estimate.  Both  growth  models  were  far  more  accurate  in  terms  of 
estimating  the  true  system  reliability  and  the  variance  associated  with  the  model  esti- 
mates was  less  than  that  produced  by  the  single  phase  MLE. 

The  MLEFD  is  particularly  sensitive  to  the  choice  of  failure  discounting  procedure 
and  parameters.  This  sensitivity  is  made  use  of  in  order  to  allow  the  model  to  track 
changing  reliability.  However,  this  property  also  places  a  premium  on  the  ability  of  the 
test  engineer  to  correctly  select  the  proper  discounting  method  and  parameters. 

The  MLEFD  did  very  well  when  the  actual  system  reliability  was  constant.  Con- 
stant system  reliability  is  seldom  present  in  actual  reliability  testing,  however.  The 
MLEFD  also  did  well  when  the  true  reliability  was  increasing  at  a  constant  rate.  Al- 
though the  exponential  regression  model  more  closely  followed  the  actual  reliability 
growth  pattern,  the  MLEFD  would  be  preferred  in  this  instance  because  of  its  smaller 
variability.  The  MLEFD  did  best  with  these  patterns  when  the  standard  discount 
method  was  used  with  the  discount  fraction  equal  to  0.25  and  the  discount  interval  equal 
to  6  or  15. 
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The  exponential  regression  model,  although  more  variable  in  the  early  stages  of 
testing,  demonstrated  the  ability  to  accurately  track  each  of  the  eight  reliability  growth 
patterns  evaluated,  even  without  the  use  of  failure  discounting.  This  model  is  also  much 
less  sensitive  to  the  choice  of  discount  method  although  its  performance  can  be  some- 
what enhanced  through  failure  discounting.  The  MLLTD,  on  the  other  hand,  failed  to 
track  the  decline  in  reliability  in  Pattern  II. 

Therefore,  the  exponential  regression  model  is  recommended  for  situations  where 
a  proven  means  of  failure  discounting  does  not  exist.  If  one  is  confident  of  the  failure 
discounting  methodology  then  the  MLEFD  may  be  more  advantageous  due  to  the  sta- 
bility o[  the  estimates  produced  by  it.  Regardless  of  the  degree  of  confidence  one  has  in 
the  failure  discounting  procedures,  the  exponential  regression  model  is  recommended  if 
abnormal  reliability  growth  is  anticipated.  Abnormal  reliability  growth  can  be  defined 
as  a  reliability  pattern  that  has  periods  of  decreasing  or  constant  reliability  in  the  middle 
of  a  generally  increasing  growth  rate.  The  exponential  regression  model  demonstrated 
superiority  to  the  MLEFD  in  its  ability  to  track  differing  patterns  of  growth. 

The  Lloyd  discount  method,  in  its  original  form,  caused  both  models  to  overesti- 
mate the  actual  system  reliability.  Therefore,  use  of  this  method  with  these  two  reliability 
growth  models  is  not  recommended.  A  modified  version  of  the  Lloyd  method  which 
employs  a  discount  interval  demonstrated  superior  performance  in  some  patterns  to  the 
original  Lloyd  discounting  method. 

C.      RECOMMENDATIONS  FOR  FURTHER  STUDY 

The  following  are  recommendations  for  further  study  and  possible  improvement  to 
the  models  introduced  in  the  study: 

•  This  study  addressed  only  discrete  reliability  growth  models.  A  similar  study  ana- 
lyzing the  characteristics  of  continuous  reliability  growth  models  should  be  con- 
ducted. This  would  necessitate  the  construction  of  a  different  simulation. 

•  Currently,  the  failure  discounting  mechanism  used  in  the  simulation  employed  in 
this  analysis  only  allows  the  user  to  set  parameter  values  at  the  beginning.  This 
should  be  modified  so  that  discount  parameters  can  be  changed  during  the  course 
of  the  simulation. 

•  The  failure  discounting  mechanism  should  be  modified  so  that  different  parameters 
can  be  applied  to  different  failure  causes.  It  may  be  desired  that  system  failures 
caused  by  failure  cause  A,  for  example,  be  discounted  less  than  those  caused  by 
failure  cause  B.  This  is  not  possible  with  the  current  simulation. 

•  Further  analysis  is  required  of  the  modified  Lloyd  failure  discounting  method. 
Many  more  simulation  runs  with  varying  reliability  growth  patterns  and  different 


53 


combinations  of  discount  parameters  are  required  before  any  solid  conclusions  can 
be  reached. 

•  A  weighted  least  squares  method  that  weights  recent  data  more  heavily  than  pre- 
vious data  should  be  analyzed  to  determine  its  ability  to  track  reliability  growth 
patterns  with  sudden  changes  in  slope  or  direction. 
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APPENDIX  A.      DERIVATION  OF  THE  EXPONENTIAL  REGRESSION 

MODEL 

This  model  was  developed  by  H.  ChernoiF  and  W.M.  Woods.  The  derivation  has 
been  presented  in  previous  works,  most  recently  by  W.M.  Woods  in  a  paper  entitled 
"Reliability  Growth  Models"  and  by  Captain  James  Drake  in  a  thesis  entitled  "Discrete 
Reliability  Growth  Models  using  Failure  Discounting".  The  exponential  regression 
model  is  based  on  the  exponential  single  phase  reliability  estimate.  The  exponential  sin- 
gle phase  estimate  may  be  expressed  mathematically  as  follows: 

R  =  1  -  e~A 

The  exponential  regression  model  uses  linear  regression  to  estimate  A  so 

A  =  a  +  fik 

where  k  is  the  testing  phase  being  used  to  compute  system  reliability.    Thus,  the  expo- 
nential regression  model  has  the  following  formula: 


Lt 


R  =  I  -  e-(*+flk)     ,  k  =   1,2,3,... 

The  estimates,  ak  and  f}k,  for  a  and  /?  at  the  end  of  the  kth  phase  are  obtained  using 
the  techniques  of  linear  regression  and  an  unbiased  estimate  for  (a  +  /?/<)  .  Let  F  denote 
the  total  number  of  failures  possible  during  the  kth  phase  and  let  j  equal  the  failure 
number  in  phase  k  such  that  7  =  1,2,3... .,F .  Njk  equals  the  number  of  trials  between  the 
(j-l)st  failure  and  the  jth  failure,  including  the  jth  failure,  in  the  kth  phase. 

An  unbiased  estimate  of  (a  +  /i/c)  using  the  jth  set  of  trials  in  phase  k  is  given  by: 

*}/<=  l  +  t  +  -  +    V  l_  ;     ifXjk  ^  2 

L  iVy/j  1 

If  .V.,  =    1  (the  first  test  was  a  failure)  then  Yjk  is  set  equal  to  zero.  The  least  squares 
estimates,  c/.k  and  fik,  for  a  and  /i  at  the  kth  phase  are  then: 

k  k 

fik  =  [Yd  -  K)YJ    -     [£(/  -  K)21 

j'=0  i=0 
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*k  =  Y  -  ftkK 

where, 

y.  =  {Yu+Y2l  +  ...+  YFi)IF 

K  =  (I  +  2  +  ...  +  k)jk 

Y  =  (Yl  +  Y2  +  ...+  Yk)lk 

Substituting  these  estimates  for  ak  and  fik  into  the  reliability  equation  yields  the  following 
estimate  of  reliability  for  phase  k: 


Rk  =   I  -  e-h  +  Mfor  k 


> 
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APPENDIX  B.      FORTRAN  CODE  FOR  SIMULATION 


-  j-  y-  j-  j-  y-  y-  j-  j«  j.  .j*  j«  j.  j^  j.  jl  .•„  j-  ju  j*  j*  j_  j«  jl  jm  j,  ju  J-  JU  -j- 


DISCRETE   RELIABILITY  GROWTH  SIMULATION 
PROGRAMMED   BY  JAMES  E  DRAKE 
MODIFIED  BY  JAMES   D  CHANDLER 
LAST  MODIFIED   29   FEB    1988 


*  THE   FOLLOWING  EXTERNAL  FILES   ARE   USED   BY  THE   PROGRAM 

*  INPUT     :    DATA  AND  PARAMETER   INPUT  FILE   (DEVICE  #    10) 

*  THESIS    :    OUTPUT  FILE   CONTAINING   INTERMEDIATE   COMPUTATIONS 

(DEVICE   #20) 

*  RELIAB:    OUTPUT  FILE   CONTAINING  FINAL  RESULTS   OF  THE   SIMULATION 

*  (DEVICE   #   30) 

*  EST  :    OUTPUT  FILE   CONTAINING   EACH  PHASE   ESTIMATE   FOR   EACH 

*  REPLICATION  OF  THE  WOODS  WEIGHTED  AVERAGE   ESTIMATE 

*  (DEVICE  #  40) 

*  MLEWD      :    OUTPUT  FILE  CONTAINING  MLE  ESTIMATES  USING  DISCOUNTING 

*  FOR  EACH  PHASE  AND  EACH  REPLICATION 

*  (DEVICE  #50) 

*  MLESP      :    OUTPUT  FILE   CONTAINING  MLE  ESTIMATE  FOR  EACH  SINGLE   PHASE 

*  AND  ALL  REPLICATIONS   USING  NO  DISCOUNTING 
(DEVICE  #60) 

*  REGEST  :    OUTPUT  FILE   CONTAINING  EACH  PHASE   ESTIMATE   FOR  EACH 

REPLICATION  OF  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

*  (DEVICE   #   70) 

*  THE  FOLLOWING   IS  A  LIST  OF  KEY  ARRAYS   USED   IN  THE   SIMULATION 

*  A  :    MAIN  WORKING  ARRAY  CONTAINS   PROBABILITY  OF   SUCCESS   FOR 

EACH  FAILURE   CAUSE,    NUMBER  OF  TRIALS   UNTIL  FAILURE   FOR 

*  EACH  FAILURE   CAUSE  AND  THE   SYSTEM,    CAUSE   OF  FAILURE, 
PHASE   NUMBER,    ADJUSTED  NUMBER  OF  TRIALS   AND  ADJUSTED 

*  NUMBER   OF  FAILURES 

DIMENSION   (    ((2*//CAUSES)+7), //FAILURES    ) 

*  NFAPH      :    CONTAINS  THE  NUMBER  OF  FAILURES    IN  EACH  PHASE 

DIMENSION   (1, //PHASES) 

*  NFCAUS   :    BINARY  ARRAY  USED  TO  DETERMINE    IF  A  FAILURE  OCCURRED   IN 

A  PHASE 

*  DIMENSION   (    1, //FAILURE   CAUSES) 

*  NTRIAL  :    CONTAINS  THE  NUMBER  OF  TRIALS   SINCE   LAST  FAILURE  OR 

*  DISCOUNTING  FOR  EACH  FAILURE   CAUSE 

*  DIMENSION   (    1, //FAILURE   CAUSES    ) 

*  PUREST  :    RECORDS   THE   PHASE   ESTIMATE  FOR  EACH  ESTIMATOR  WITHIN  A 

SINGLE   REPLICATION 

*  DIMENSION   (4, //PHASES) 

*  ROW   1   :    WOODS  WEIGHTED  AVERAGE  ESTIMATE 

*  ROW  2   :    MLE  WITH  DISCOUNTING 


* 
ft 
* 

* 

ft 
* 
ft 

ft 


ft 


ft 
ft 


it 
* 

* 
ft 

ft 


ft 


ft 
ft 
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A 

A 


A 

A 

* 

A 

A 

A 


AREL 

YJK 

CUMSF 


REG 


ROW  3  :  SINGLE  PHASE  MLE 

ROW  4  :  EXPONENTIAL  REGRESSION  ESTIMATE 

CONTAINS  ACTUAL  SYSTEM  RELIABILITY  IN  EACH  PHASE 

DIMENSION  (1, //PHASES) 

CONTAINS  YJK  VALUES  UP  TO  1000 

DIMENSION  (1,1000) 

CONTAINS  THE  NUMBER  OF  SUCCESS 

FAILURE  CAUSE  (USED  WITH  WOODS 

DIMENSION  (3, //FAILURE  CAUSES) 


AND  FAILURES  FOR  EACH 
WEIGHTED  AVERAGE  EST. ) 


ROW  1 
ROW  2 
ROW  3 
ARRAY 


NUMBER  OF  FAILURES 
NUMBER  OF  SUCCESSES 
ADJUSTED  NUMBER  OF  SUCCESSES 
USED  TO  COMPUTE  THE  EXPONENTIAL 


REGRESSION  ESTIMATE 


DIMENSION 


POW 
ROW 
ROW 
ROW 
ROW 


(5, //PHASES) 

BAR 

BAR 

BAR  FOR  THE 

HAT 

HAT 


A 
A 
A 

A 
A 
A 
A 

A 

* 
A 
A 

A 


w 

A 
A 
A 

A 


THE  REMAINING  ARRAYS  ARE  USED  TO  COMPUTE  THE  MEAN  AND  VARIANCE 
OF  EACH  ESTIMATE  AT  EACH  PHASE.  THEY  ALL  HAVE  THE  SAME  DIMENSIONS 
AND  STRUCTURE 

DIMENSION  (4, //PHASES) 


EST 

MLEWD 
MLESP 
REGEST 


ROW 
ROW 
ROW 
ROW 


RUNNING  SUM  OF  ESTIMATES 

RUNNING  SUM  OF  SQUARED  ESTIMATES 

MEAN  OF  THE  ESTIMATES 

STANDARD  DEVIATION  OF  THE  ESTIMATES 


VALUES  FOR  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

VALUES  FOR  THE  MLE  WITH  DISCOUNTING 

VALUES  FOR  THE  SINGLE  PHASE  MLE 

VALUES  FOR  THE  EXPONENTIAL  REGRESSION  ESTIMATE 


A 
A 
A 
A 
A 
A 
A 
A 
A 


VAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAiVAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 


C  DEFINE  AND  DIMENSION  VARIABLES 

PARAMETER  (NR=50 ,NC=200) 

INTEGER  REP,CUMSF,DISOPT,FRELOP,LDI,ALD 

REAL*4  MIN 

REAL*8  DSEED, MLESP, MLEWD, EST, EUL 

DIMENSION  NFAPH(NR) ,A(NR,NC) ,NFCAUS(NR) ,NTRIAL(NR) ,PHREST(4,NR) ,ES 
CT(4,NR) ,MLEWD(4,NR) ,MLESP(4,NR) ,REGEST(4,NR) ,AREL(NR) ,YJK( 1000) ,CU 
CMSF(3,NR),REG(5,NR) 

C  READ  IN  THE  NUMBER  OF  CAUSES  TO  BE  USED  (  NCAUSE  )  AND  THE  NUMBER 
C   OF  PHASES  (  NPHASE  )  IN  THE  TEST 

READ(10,*)  NCAUSE 
READ(10,*)  NPHASE 

C  CHECK  IF  FIXED  RELIABILITY  OPTION  IS  DESIRED.  FIX  EULER'S  NUMBER. 
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READ(10,*)  FRELOP 
EUL  =  0.5772156648 

C  CREATE  VARIABLES  FOR  THE  ROW  INDICES  OF  THE  WORKING  MATRIX  (  A  ) 

C  IPHASE:  PHASE 

C  ISYSPR:  ACTUAL  COMPONENT  RELIABILITY 

C  INTR:  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 

C  IFAILC:  CAUSE  OF  THE  FAILURE 

C  IADJF:  ADJUSTED  NUMBER  OF  FAILURES  ED 

C  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 

C  IADJT:  ADJUSTED  NUMBER  OF  TRIALS  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 

C  IYJK:  YJK  COMPUTED  ON  THE  ADJUSTED  NUMBER  OF  TRIALS 

IPHASE  =  (2*NCAUSE)+1 
ISYSPR  =  IPHASE  +1 
INTR  =  ISYSPR  +  1 
IFAILC  =  INTR  +  1 
IADJF  =  IFAILC  +  1 
IADJT  =  IADJF  +  1 
IYJK  =  IADJT  +  1 

C  READ  IN  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE  (  NFAPH(I)  )  AND 
C   COMPUTE  THE  TOTAL  NUMBER  OF  FAILURES  IN  THE  TEST  (  NFAIL  ) 

NFAIL  =  0 

DO  10  I=1,NPHASE 

RE AD (10,*)  NFAPH(I) 

NFAIL  =  NFAIL  +  NFAPH(I) 
10   CONTINUE 

C   INPUT  THE  PROBABILITY  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C   IN  EACH  PHASE  IF  FRELOP  EQUALS  ONE. 

IF  (FRELOP  . EQ.  1)  THEN 
DO  15  I=1,NCAUSE 
DO  15  J=l, NFAIL 
READ(10,*)  A(I,J) 
15     CONTINUE 
ELSE 

C   INPUT  THE  PROBABILITY  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C   IN  THE  FIRST  PHASE  IF  FRELOP  EQUALS  ZERO. 

DO  20  I=1,NCAUSE 

READ(10,*)  A(I,1) 
20   CONTINUE 

ENDIF 

C  INPUT  THE  REMAINING  VARIABLES  ,  THE  NUMBER  OF  SUCCESSFUL  TRIALS 

C  BEFORE  A  DISCOUNT  IS  APPLIED  (N);  THE  DISCOUNT  FACTOR  (R);  THE  SEED 

C  FOR  THE  RANDOM  NUMBER  GENERATOR,  GGUBFS,  (DSEED);  RELIABILITY 

C  GROWTH  FRACTION  (FRIMP);  TRIGGER  FOR  PRINTING  INTERMEDIATE  OUTPUT 

C  (IOPT) 
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TRIGGERS  FOR  SAVING  EACH  ESTIMATE  AT  EACH  PHASE  FOR  EACH  ESTIMATOR 

IOPT1  :  WOODS  WEIGHTED  AVERAGE  MODEL 

IOPT2  :  MLE  WITH  DISCOUNTING 

IOPT3  :  SINGLE  PHASE  MLE 

I0PT4  :  EXPONENTIAL  REGRESSION  MODEL 
DISCOUNTING  OPTION  TRIGGER  (DISOPT);  LLOYD  FAILURE  DISCOUNTING 
PARAMETER  (GAMMA);  LLO^D  DISCOUNT  INTERVAL 


READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 
READ( 


10,*) 
10,*) 

10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 
10,*) 


N 

R 

DSEED 

FRIMP 

MREP 

IOPT 

I0PT1 

I0PT2 

I0PT3 

I0PT4 

DISOPT 

GAMA 

LDI 


XNREP  =  NREP 
DSEED1  =  DSEED 

C   INITIALIZE  THE  ARRAYS  USED  TO  COMPUTE  THE  MEAN  AND  STANDARD  DEVIATION 
C  OF  EACH  ESTIMATOR 

DO  30  J=1,NPHASE 
DO  30  1=1,4 

EST(I,J)  =  0. 0 
MLEWD(I,J)  =  0.  0 
MLESP(I,J)  =0.0 
REGEST(I,J)  =  0. 0 
PHREST(I,J)  =  0. 0 
30   CONTINUE 

C  COMPUTE  AND  STORE  THE  YJK  VALUES  UP  TO  1000 

YJK(l)  =0.0 
DO  40  1=1,999 

YJK(I+1)  =  YJK(I)  +  1.0/1 
40   CONTINUE 

C  COMPUTE  AND  STORE  K  BAR  FOR  THE  EXPONENTIAL  REGRESSION  MODEL 

SUM  =  0.0 
DO  50  I=1,NPHASE 
SUM  =  SUM  +  I 
REG(1,I)  =  SUM/ I 
50   CONTINUE 

C   MAJOR  REPETITION  OF  THE  SIMULATION  LOOP 

DO  500  REP=1,NREP 
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C   INITIALIZE  FAILURE  CAUSE  VECTOR  (NFCAUS)  AND  (CUMSF) 
C   COMPUTE  THE  INITIAL  SYSTEM  RELIABILITY 

REL  =  1. 

DO  60  I=1,NCAUSE 

NFCAUS(I)  =  0 

REL  =  REL  *  A(I,1) 

DO  60  J=l,3 

CUMSF(J,I)  =  0 
60   CONTINUE 

C   INITIALIZE  COLUMN  (FAILURE  #  )  COUNTER  FOR  THE  WORKING  ARRAY  (A) 

J  =  1 

C   LOOP  TO  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 
C  AND  THE  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  EACH  PHASE 

DO  130  K=1,NPHASE 

C   SKIP  ACTUAL  COMPONENT  RELIABILITY  COMPUTATION  AFTER  FIRST  REP 
C  AND  FOR  FIRST  FAILURE 

IF(J.EQ.  1)  GOTO  75 
IF(REP.  GT.  1)  GOTO  75 
REL  =  1. 

C   IF  FIXED  RELIABILITY  OPTION  IS  SELECTED  THEN  PHASE  RELIABILITIES 
C  ARE  COMPUTED  AS  FOLLOWS 

IF  (FRELOP  .EQ.  1)  THEN 
DO  65  I=1,NCAUSE 
REL  =  REL-A(I,J) 
NFCAUS(I)  =  0 
65       CONTINUE 
ELSE 

C   COMPUTE  NEW  ACTUAL  RELIABILITY  FOR  THE  COMPONENT  IN  PHASE  K 

DO  70  I=1,NCAUSE 

C   INCREASE  CAUSE  PR( SUCCESS)  IF  IT  CAUSED  FAILURE  IN  THE  PREVIOUS  PHASE 
C  COMPUTE  NEXT  PHASE  RELIABILITY  AND  REINITIALIZE  NFCAUS  (NOT  USED  IF 
C   FIXED  PHASE  RELIABILITY  OPTION  IS  SELECTED). 

IF( NFCAUS ( I). EQ.  1)  THEN 

A(I,J)  =  A(I,(J-1))  +  ((1.  -  A(I,(J-1)))*FRIMP) 
ELSEIF(NFCAUS(I).NE.  1)  THEN 
A(I,J)  =  A(I,(J-1)) 
ELSE 
ENDIF 

REL  =  REL»A(I,J) 
NFCAUS(I)  =  0 
70     CONTINUE 
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ENDIF 

75     Jl  =  1 

TRTOT  =  0.  0 

C   COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE  AND  THE 
C   CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  THE  PHASE 

DO  120  L=l,NFAPHfK) 

IF(REP.GT.  1)  GOTO  90 

IF(J1.EQ.  1)  GOTO  85 

IF  (FRELOP  .  EQ.  1)  GOTO  85 

DO  80  I=1,NCAUSE 

A(I,J)  =  A(I,(J-1)) 
80       CONTINUE 
85       A(ISYSPR,J)  =  REL 

A(IPHASE,J)  =  K 
90       MIN  =  7.  2E75 

DO  110  I=1,NCAUSE 

C   ASSIGN  #  TRIALS  FOR  CAUSES  WITH  PR( SUCCESS)  =  0  OR  1 

IF(A(I,J).GE.  1.  )  THEN 

A((I+NCAUSE),J)  =  7. 2E75 

GOTO  100 
ELSEIF(A(I,J).EQ.  0.  )  THEN 

A((I+NCAUSE),J)  =  1. 

GOTO  100 
ELSE 
ENDIF 

C  CONVERT  UNIFORM  (0,1)  RANDOM  VARIABLE  TO  GEOMETRIC  (#  TRIALS  UNTIL 

C  FAILURE  )  FOR  EACH  FAILURE  CAUSE.  RECORD  THE  MIN  #  TRIALS  FOR  THE 

C  CAUSES  AS  THE  SYSTEM  #  TRIALS  UP  TO  AND  INCLUDING  FAILURE  AND 

C  RECORD  THE  FAILURE  CAUSE 

A((I+NCAUSE),J)  =  INT(1.+(L0G(GGUBFS(DSEED))/L0G(A(I,J)))) 
100        IF(A((I+NCAUSE),J).  LE.MIN)  THEN 
MIN  =  A((I+NCAUSE),J) 
IMIN  =  I 
ELSE 
ENDIF 
110      CONTINUE 

A(IFAILC,J)  =  IMIN 
NFCAUS(IMIN)  =  1 

C   COMPUTE  THE  TOTAL  #  OF  TRIALS  FOR  THE  MLE  SINGLE  PHASE  ESTIMATE  AND 
C   INCREMENT  FAILURE  //  COUNTERS 

A(INTR,J)  =  MIN 
TRTOT  =  TRTOT  +  A(INTR,J) 
J  =  J  +  1 
Jl  =  Jl  +  1 
120   CONTINUE 
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C   COMPUTE  THE  MLE  ESTIMATE  OF  COMPONENT  RELIABILITY  FOR  THIS  PHASE  AND 
C   COMPUTE  THE  RUNNING  SUM  OF  ESTIMATES  AND  THE  SUM  OF  ESTIMATES  SOUARED 
C   FOR  COMPUTATION  OF  THE  MEAN  AND  STANDARD  DEVIATION  OF  THE  ESTIMATE 

PHREST(3,K)  =  (TRTOT  -  NFAPH(K) )/TRTOT 
MLESP(1,K)  =  MLESP(1,K)  +  PHREST(3,K) 
MLESP(2,K)  =  MLESP(2,K)  +  (PHREST( 3 ,K)**2) 

130   CONTINUE 

C   INITIALIZE  THE  ADJUSTED  NUMBER  OF  FAILURES  TO  1  AND  THE  COUNT  OF  THE 
C  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  (NTRIALS(I)  )  TO  0 
C   IN  PREPARATION  FOR  THE  DISCOUNTING  ROUTINE 

DO  140  J=1,NFAIL 

A(IADJF,J)  =  1. 

140   CONTINUE 

DO  150  I=1,NCAUSE 
NTRIAL(I)  =  0 
150   CONTINUE 

C  DISCOUNTING  ROUTINE  REVIEWS  ALL  PAST  FAILURES  AND  CAUSES  TO  DATE 

C  AND  DETERMINES  IF  THE  DISCOUNTING  CONDITIONS  HAVE  BEEN  MET.  COMPUTES 

C  THE  ADJUSTED  FAILURES,  THE  ADJUSTED  #  OF  TRIALS  AND  YJK 

J  =  0 

DO  300  K=1,NPHASE 
DO  200  L=1,NFAPH(K) 
J  =  J  +  1 

C   UPDATES  THE  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  EACH 
C  FAILURE  CAUSE 

ICAUSE  =  INT(A(IFAILC,J)+.  5) 
DO  160  I=1,NCAUSE 

IF( ICAUSE. EQ.  I)  THEN 

NTRIAL(I)  =  0 
ELSEIF( ICAUSE.  NE.  I)  THEN 

NTRIAL(I)  =  NTRIAL(I)  +  INT( A( INTR, J)+.  5) 
ELSE 
END  IF 
160     CONTINUE 

200   CONTINUE 

C   CHOOSE  DISCOUNTING  METHOD  TO  BE  USED 

IF(DISOPT. NE. 2)  GOTO  180 

C  PERFORM  LLOYD'S  FAILURE  DISCOUNTING  METHOD 

DO  170  1=1, J 

II  =  INT(A(IFAILC,I)+. 5) 
IF(NTRIAL(I1).EQ.  0)  THEN 
A(IADJF,I)  =  1.0 
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GOTO  170 
ELSE 

ENDIF 

C  THIS  IS  THE  MODIFIED  LLOYD  METHOD  USING  A  DISCOUNT  INTERVAL.  THE 
G  ORIGINAL  DISCOUNT  METHOD  MAY  BE  EMPLOYED  BY  SETTING  LDI  TO  ONE. 

ALD  =  INT(NTRIAL(I1)/LDI) 
IF(ALD  .EQ.  0)  THEN 

A(IADJF,I)  =  1.0 

GO  TO  170 
ELSE 

A(IADJF,I)  =  1.0  -  ((1.  -GAMA)**(1.  O/ALD)) 
ENDIF 

170     CONTINUE 
GOTO  210 

C  PERFORMS  STRAIGHT  PERCENT  FAILURE  DISCOUNTING  AND 
C  COMPUTES  THE  ADJUSTED  #  OF  FAILURES 

180      DO  190  1=1, J 

II  =  INT(A(IFAILC,I)+. 5) 
IF(NTRIAL(I1).EQ.  0)  THEN 

A(IADJF,I)  =  1. 
ELSEIF(NTRIAL(I1).GE.N)  THEN 

A(IADJF,I)  =  A(IADJF,I)*((1.  -R)**(NTRIAL( Il)/N) ) 
ELSE 
ENDIF 
190      CONTINUE 

C  ADJUSTS  THE  #  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  THOSE  CAUSES 
C  THAT  HAVE  MET  OR  SURPASSED  THE  DISCOUNTING  THRESHOLD 
C  FOR  THE  STRAIGHT  PERCENT  DISCOUNTING  METHOD 

DO  205  I=1,NCAUSE 

IF(NTRIAL(I).GE.N)  NTRIAL(I)  =  MOD(NTRIAL( I) ,N) 
205      CONTINUE 
210    TADJT  =  0. 0 
TYJK  =  0.  0 
TPYJK  =  0.0 
Kl  =  0 

DO  215  12=1,3 
DO  215  I=1,NCAUSE 
CUMSF(I2,I)  =  0 
215       CONTINUE 

C   COMPUTES  THE  ADJUSTED  #  OF  TRIALS  FROM  THE  ADJUSTED  #  OF  FAILURES 
C  AND  COMPUTES  THE  SUM  OF  THE  ADJUSTED  #  OF  TRIALS  FOR  ESTIMATE  COMP. 

PREL  =  0.0 
LTRIAL  =  0 

C   IF  ADJUSTED  FAILURES  ARE  APPROACHING  0  THEN  ADJUSTED  TRIALS  MUST 
C   BE  PRE-SET. 
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DO  240  1=1, J 

IF(A(IADJF,I)  .LE.  .0000001)  THEN 
A(IADJF,I)  =  . 0000001 
ENDIF 

A(IADJT,I)  =  A(INTR,I)/A(IADJF,I) 
TADJT  =  TADJT  +  A(IADJT,I) 

C   COMPUTE  YJK  FROM  THE  ADJUSTED  #  OF  TRIALS  AND  STORE  THE  SUM  FOR 

C   ESTIMATE  COMPUTATION,  USE  ARRAY  FOR  #  TRIALS  <  1000  AND  APPROX.  FOR 

C  VALUES  >  1000 

Nl  =  NINT(A(IADJT,I)) 
IF(N1.LE.  1000)  THEN 

A(IYJK,I)  =  YJK(Nl) 
ELSEIF(N1.GT.  1000)  THEN 

X=N1 

Q=12*X 

T=X+1 

S=X+2 

A(IYJK,I)=(EUL+(L0G(X))+(1/(2*X))-(1/(Q*T))-(1/(Q*T*S))) 

ELSE 
ENDIF 

C  DETERMINE  IF  A  PHASE  BOUNDARY  HAS  BEEN  REACHED  TO  BEGIN  ESTIMATE 
C  COMPUTATION 

IF(I.EQ.  1)  GOTO  225 

IF( A( IPHASE , I ) . NE. A( IPHASE , ( I - 1 )  ) )  THEN 

C  COMPUTE  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

MAX  =  0 

Kl  =  Kl  +  1 

C  DETERMINE  THE  FAILURE  CAUSE  WITH  THE  LARGEST  #  OF  FAILURES 

DO  220  I1=1,NCAUSE 

IF(CUMSF( 1,11). GT. MAX)  THEN 
MAX  =  CUMSF(1,I1) 
ICOL  =  II 
ELSE 
ENDIF 
220   CONTINUE 

C  COMPUTE  YJK  VALUE  FOR  THE  CURRENT  PHASE  ESTIMATE 

IF(CUMSF( 1, ICOL).  LE.  1000)  THEN 

AHATL  =  YJK(CUMSF(l,ICOL)) 
ELSEIF(CUMSF(l,ICOL).GT. 1000)  THEN 

X  =  CUMSF(l,ICOL) 

Q=12*X 
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T=X+1 
S=X+2 

AHATL=(EUL+(LOG(X))+( 1/(2*X) ) -( 1/(Q*T) ) -( 1/(Q*T*S)) ) 

ELSE 
ENDIF 

IX  =  CUMSF(1,IC0L)  +  CUMSF(3,ICOL) 
IF(IX. LE. 1000)  THEN 

AHATU  =  YJK(IX) 
ELSEIF(IX.  GT.  1000)  THEN 
X  =  IX 
0=1 2*X 
T=X+1 
S=X+2 

AHATU=(EUL+(LOG(X) )+( 1/(2*X) ) -( 1/(Q*T) ) -( 1/(Q*T*S) ) ) 

ELSE 
ENDIF 

C  COMPUTE  CURRENT  PHASE  RELIABILITY  ESTIMATE 

AHAT  =  AHATU  -  AIIATL 
CREL  =  1.  0  -  EXP(-AHAT) 

X  =  CUMSF(1,IC0L)  +  CUMSF(3,IC0L) 

C  COMPUTE  AND  STORE  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

PREL  =  ((LTRIAL*PREL)/X)  +  ( ( (X-LTRIAL)*CREL)/X) 
LTRIAL  =  CUMSF(1,IC0L)  +  CUMSF( 3 , ICOL) 

C  COMPUTE  THE  PHASE  AND  GLOBAL  AVERAGE  FOR  YJK  USED  IN  THE  EXPONENTIAL 
C  REGRESSION  ESTIMATES  ARE 

REG(2,K1)  =  TYJK/(I-1) 
REG(3,K1)  =  TPYJK/NFAPH(K1) 
TPYJK  =  0.0 

ENDIF 

C  COMPUTE  THE  NUMBER  OF  FAILURES  AND  SUCCESSES  FOR  EACH  FAILURE  CAUSE 
C   USED  IN  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE 

225   ICAUSE  =  INT(A(IFAILC,I)+. 5) 
DO  230  I1=1,NCAUSE 

CUMSF(2,I1)  =  CUMSF(2,I1)  +  INT( A( INTR, I )  +  .5) 
CUMSF(3,I1)  =  CUMSF(3,I1)  +  Nl 
230   CONTINUE 

CUMSFC1, ICAUSE)  =  CUMSF( 1 , ICAUSE)  +  1 
CUMSF( 2, ICAUSE)  =  CUMSF( 2 , ICAUSE)  -  1 
CUMSF( 3, ICAUSE)  =  CUMSF( 3 , ICAUSE)  -  1 
TPYJK  =  TPYJK  +  A(IYJK,I) 
TYJK  =  TYJK  +  A(IYJK,I) 
240    CONTINUE 
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C  REPEAT  COMPUTATIONS  FOR  THE  WOODS  WEIGHTED  AVERAGE  ESTIMATE  FOR  THE 
C   FINAL  PHASE 

MAX  =  0 

Kl  =  Kl  +  1 

DO  245  I1=1,NCAUSE 

IF(CUMSF( 1,11).  GT.  MAX)  THEN 

MAX  =  CUMSF(1,I1) 

ICOL  =  II 

ELSE 

END  IF 
245   CONTINUE 

IF(CUMSF(l,ICOL).LE.  1000)  THEN 

AHATL  =  YJK(CUMSF(l,ICOL)) 
ELSEIF(CUMSF(l,ICOL).GT. 1000)  THEN 

X  =  CUMSF(l,ICOL) 

Q=12*X 

T=X+1 

S=X+2 

AHATL=(EUL+(LOG(X) )+( 1/(2*X) ) -( 1/(Q*T) ) -( 1/(Q*T*S)) ) 

ELSE 
ENDIF 

IX  =  CUMSF(l,ICOL)  +  CUMSF(3,ICOL) 
IF(IX. LE. 1000)  THEN 

AHATU  =  YJK(IX) 
ELSEIF(IX. GT. 1000)  THEN 
X  =  IX 
Q=12*X 
T=X+1 
S=X+2 

AHATU=( EUL+( LOG( X) )+( l/( 2*X) ) -( 1/(Q*T) ) -( 1/(Q*T*S) ) ) 

ELSE 
ENDIF 

AHAT  =  AHATU  -  AHATL 
CREL  =  1.0  -  EXP(-AHAT) 

X  =  CUMSF(l,ICOL)  +  CUMSF(3,ICOL) 

PREL  =  ((LTRIAL*PREL)/X)  +  (( (X-LTRIAL)*CREL)/X) 
LTRIAL  =  CUMSF(l,ICOL)  +  CUMSF(3 , ICOL) 
REG(2,K1)  =  TYJK/(J) 
REG(3,K1)  =  TPYJK/NFAPH(K1) 

PHREST(l.K)  =  PREL 
C  COMPUTE  THE  MLE  ESTIMATE  OF  PHASE  RELIABILITY  USING  DISCOUNTING 

PHREST(2,K)  =  (TADJT  -  J)/TADJT 
C   COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE  BEGINNING  WITH  B  HAT 
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SUM  =  0.  0 

SUMS  =  0.0 

IF  (K.  EQ.  1)  GOTO  252 

DO  250  I  =  1,K 

SUM  =  SUM  +  ((I-REG(1,K))*REG(3,I)) 

SUMS  =  SUMS  +  ((I-REG(1,K))**2) 
250     CONTINUE 

REG(4,K)  =  SUM/SUMS 
C  COMPUTE  A  HAT 

REG(5,K)  =  REG(2,K)  -  (REG(4,K)*REG(1,K)) 

C  COMPUTE  AND  STORE  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

PHREST(4,K)  *  1.0  -  EXP( -(REG(5 ,K)  +  (REG(4,K)*K) ) ) 
IF(PHREST(4,K).  LT.  0.  0)  PHREST(4,K)=0.  0 
GOTO  255 
252     PHREST(4,K)  =  1.0-  EXP( -REG(  3  , 1) ) 

IF(PHREST(4,K).  LT. 0. 0)  PHREST(4,K)=0. 0 

C  STORE  THE  RUNNING  SUM  OF  THE  ESTIMATES  FOR  THE  CURRENT  PHASE  AND  THE 

C  RUNNING  SUM  OF  THE  ESTIMATES  SQUARED  FOR  COMPUTATION  OF  THE  MEAN  AMD 

C  STANDARD  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  RELIABILITY  GROWTH 

C  MODEL 

255    EST(1,K)  =  EST(1,K)  +  PHREST(1,K) 

EST(2,K)  =  EST(2,K)  +  (PHREST( 1 ,K)**2) 
MLEWD(1,K)  =  MLEWD(1,K)  +  PHREST(2,K) 
MLEWD(2,K)  =  MLEWD(2,K)  +  (PHREST( 2,K)**2) 
REGEST(1,K)  =  REGEST(1,K)  +  PHREST(4,K) 
REGEST(2,K)  =  REGEST(2,K)  +  (PHREST(4,K)**2) 

C   STORE  THE  ACTUAL  PHASE  RELIABILITY 

AREL(K)  =  A(ISYSPR,J) 

C  PRINT  INTERMEDIATE  OUTPUT  IF  REQUESTED  AND  THE  NUMBER  OF  REPETITIONS 
C   IS  NOT  GREATER  THAN  5 

IF(IOPT. NE.  1)  GOTO  300 
IF(REP.  GT.  5)  GOTO  300 

WRITE(20,1000)  REP,K 
1000  FORMAT(T16,' REPETITION  NUMBER:  ',14,'    PHASE  NUMBER:  ',14) 

WRITE(20,1010)  A(ISYSPR,J) 
1010  FORMAT(22X, 'ACTUAL  COMPONENT  RELIABILITY:  ',F7.5) 

WRITE(20,1020)  PHREST(1,K) 
1020  FORMAT(20X, 'PREDICTED  COMPONENT  RELIABILITY:  *,F7.5) 

WRITE(20,1022^  PHREST(2,K) 
1022  FORMAT(20X,'MLE  ESTIMATE  USING  DISCOUNTING:  *,F7.5) 

WRITE(20,1025)  PHREST(3,K) 
1025  FORMAT(18X,'MLE  ESTIMATE  OF  PHASE  RELIABILITY:  \F7.5) 

WRITE(20,1027)  PIIREST(4,K) 
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1027  FORMATC 14X,' REGRESSION  ESTIMATE  OF  PHASE  RELIABILITY:  ',F7.5) 

WRITE(20,1030) 
1030  FORMAT( '  ' , '      ' ) 

DO  260  I=1,NCAUSE 

WRITE(20, 1035)1, A( I   J) , A( ( I+NCAUSE) , J) 

1035  FORMAT(12X,'  CAUSE:     f,I3,'         PR(  SUCCESS):     ',F7.6,'        //TRIALS:     \ 
CF10.  0) 

260      CONTINUE 

WRITE(20,1036) 

1036  FORMAT( '  ' , '      ' ) 
WRITE(20,1040) 

1040  F0RMAT(4X,'FAIL  #',3X,'FAIL  CAUSE', 3X,*#  TRIALS ' , 3X, ' ADJ  #  FAIL', 3 
CX,'ADJ  #  TRIALS' ,7X,'YJK') 
DO  270  1=1, J 

WRITE(20,1050)I,A(IFAILC,I),A(INTR,I),A(IADJF,I),A(IADJT,I),A(IYJK 
C,I) 
1050  FORMATC 4X, 13, 8X,F3. 0, 7X,F8.  0 ,4X,F3.  6 ,4X,F12.  0 , 3X,F11.  4) 
270   CONTINUE 

WRITE(20,1060) 
1060  FORMATC'  ',///) 

300   CONTINUE 
C   PRINT  EACH  OF  THE  3  ESTIMATES  TO  THEIR  APPROPRIATE  OUTPUT  FILE 
C   IF  REQUESTED 

IF(I0PT1.NE.  1)  GOTO  401 

400  WRITE(40,2000)  (PHRESTC 1, I) ,  I=1,NPHASE) 

401  IF(I0PT2.NE. 1)  GOTO  402 

WRITE (5 0,2000)  (PHREST(2, I) ,  I=1,NPHASE) 

402  IF(I0PT3.NE.  1)  GOTO  403 
WRITE(60,2000)  (PHREST( 3, I) ,  I=1,NPHASE) 

403  IF(I0PT4.NE.  1)  GOTO  500 
WRITE(70,2000)  ( PHREST(4, I) ,  I=1,NPHASE) 

2000  FORMATC*  ' , 30(F7. 6: IX) ) 

500   CONTINUE 

C  UPON  COMPLETION  OF  ALL  REPETITIONS,  COMPUTE  THE  MEAN  AND  STANDARD 
C  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  PHASE  SKIPPING  COMPUTATIONS  IF 
C   ONLY  ONE  REPETITION  IS  REQUIRED 

IF  (NREP. LE.  1)  GOTO  601 

DO  600  I=1,NPHASE 
EST(3,I)  =  EST(1,I)/XNREP 
MLEWD(3,I)  =  MLEWD(1,I)/XNREP 
MLESP(3,I)  =  MLESP(1,I)/XNREP 
REGEST(3,I)  =  REGEST(1,I)/XNREP 

EST(4,I)  =  SQRT((EST(2,I)-(XNREP*(EST(3,I)**2)))/(XNREP-1)) 
MLEWD( 4 , I )  =  SQRT( (MLEWD( 2 , I ) -( XNREP* ( MLEWD( 3 , I)**2) ) ) /(XNREP- 1 ) ) 
MLESP( 4 , I )  =  SQRT( ( MLESP( 2 , I ) - ( XNREP* ( MLESP( 3,1) **2 ) ) ) / ( XNREP- 1 ) ) 
REGEST( 4 , I )=SQRT( ( REGEST( 2 , I ) - ( XNREP- ( REGEST( 3 , I )**2 ) ) ) / ( XNREP- 1 ) ) 
600   CONTINUE 

C  PRINT  THE  FINAL  OUTPUT  TABLE  TO  A  FILE 
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601  WRITEf 30,3000) 

3000  FORMATC'O' ,T47,' DISCRETE  RELIABILITY  GROWTH  SIMULATION') 

WRITE( 30,3010) 
3010  FORMATC'-' ,T54, 'MODEL  PARAMETER  SUMMARY') 

WRITE ( 30,3020)  NCAUSE 
3020  FORMATC'O' ,T4 7,' NUMBER  OF  POSSIBLE  FAILURE  CAUSES  ',14) 

IF  (FRELOP  . EQ.  1)  GOTO  4000 

WRITE(30,3030) 
3030  FORMAT('0' ,T38, 'CAUSE  NUMBER' ,T64, ' SINGLE  TRIAL  PR(  SUCCESS  )  FOR 
CPHASE  1') 

DO  3050  M=l, NCAUSE 

WRITE(30,3040)  M,A(M,1) 
3040  FORMATC'  ' ,T43,I2,T79,F8. 6) 
3050  CONTINUE 

WRITE(30,3060)  FRIMP 
3060  FORMATC'O' ,T3 7,' FRACTION  CAUSE  RELIABILITY  IMPROVES  AFTER  FAILURE 

C' ,F8. 6) 
5000  WRITEC30,3080)  NPHASE 
3080  FORMATC '-' ,T48,' NUMBER  OF  PHASES  IN  THE  SIMULATION  ',12) 

WRITE(30,3090) 
3090  FORMATC'O' ,T42,' PHASE  NUMBER' ,T59 , 'NUMBER  OF  FAILURES  IN  THE  FIRST 
C  PHASE') 

DO  3110  M=l, NPHASE 

WRITE(30,3100)  M,NFAPH(M) 
3100  FORMATC'  ' ,T43 , I2,T73, 12) 
3110  CONTINUE 

WRITEC30,3120)  NFAIL 
3120  FORMATC'O' ,T51, 'TOTAL  NUMBER  OF  FAILURES  ',14) 

IFCDISOPT.  EQ.  2)  GO  TO  3160 

WRITEC30,3130) 
3130  FORMATC'-' ,T38,' DISCOUNTING  PERFORMED  USING  THE  CONSTANT  FRACTION 
CMETHOD') 

WRITE(30,3140)  R 
3140  FORMATC'O' ,T44, 'FRACTION  EACH  FAILURE  IS  DISCOUNTED  ' ,F8. 6) 

WRITE(30,3150)  N 
3150  FORMATC'  ' ,T33 ,' NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  DISCOUNT 
C  IS  APPLIED  ' ,14) 

GO  TO  3190 
3160  WRITE(30,3170) 
3170  FORMATC'-' ,T44,'D I SCOUNTING  PERFORMED  USING  THE  LLOYD  METHOD') 

WRITEC 30,3180)  GAMA 
3180  FORMATC'O' ,T39,' PERCENT  C.I.  C  USED  AS  DISCOUNT  FRACTION  )  ' ,F8. 6 
C) 

WRITEC 30, 3185)  LDI 
3185  FORMATC'O' ,T50,' LLOYD  DISCOUNT  INTERVAL:  *,I3) 
3190  WRITEC 30, 3200)  DSEED1 
3200  FORMATC'-' ,T46,' RANDOM  NUMBER  SEED  USED  ' ,F15. 2) 

WRITEC 30, 3210)  NREP 
3210  FORMATC'O' ,T37,' NUMBER  OF  REPETITIONS  OF  THE  SIMULATION  PERFORMED 
C',I7) 

WRITEC 30, 3220) 
3220  FORMATC ' 1 ' ,T61 , ' ESTIMATOR:  ' ) 

WRITEC 30,3230) 
3230  FORMATC '0* ,T48,' SINGLE  PHASE  MLE  WITHOUT  DISCOUNTING') 

WRITEC30,3240) 
3240  FORMATC'-' ,T60,' MEAN*  ,T83,' ESTIMATE ' ,T109,' 95  V  ) 
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WRITE (30, 3250) 

3250  FORMATC '  ' ,T12, 'PHASE  NUMBER' ,T29, 'ACTUAL  RELIABILITY' ,T52, 'FPEDIC 
CTED  RELIABILITY' ,T7 8, 'STANDARD  DEVIATION' ,T101, 'CONFIDENCE  INTERVA 
CL') 

C   COMPUTE  C. I.  FOR  SINGLE  PHASE  MLE 

DO  3270  M=I,NPHASE 

CI  =  (1. 96*MLESP(4,M))/SQRT(XNREP) 

CIU  =  MLESP(3,M)  +  CI 

OIL  =  MLESP(3,M)  -  CI 

WRITE(30,3260)  M, AREL(M) ,MLESPf 3 ,M) ,MLESP(4 ,M) ,CIL,CIU 
3260  FORMATC  0'  ,T17 , 12 ,T34,F8.  6 ,  T58,F8. 6,T82,F9.  6,  T99,*(  ',F8.6,'  ,  '  ,F 

C8.6,'  )') 
3270  CONTINUE 

WRITE(30,3220) 

WRITE(30,3280) 
3280  FORMATC 0' ,T42,' MAX  LIKELIHOOD  ESTIMATE  USING  DISCOUNTED  FAILURES' 
C) 

WRITE( 30,3240) 

WRITE(30,3250) 

C   COMPUTE  C. I.  FOR  MLE  WITH  DISCOUNTING 

DO  3290  M=1,NPHASE 

CI  =  (1.96*MLEWD(4,M))/SQRT(XNREP) 

CIU  =  MLEWD(3,M)  +  CI 

CIL  =  MLEWD(3,M)  -  CI 

WRITE(30,3260)  M,AREL(M) ,MLEWD(3,M) ,MLEWD(4,M) , CIL, CIU 
3290  CONTINUE 

WRITE(30,3220) 

WRITE(30,3300) 
3300  FORMATC 0' ,T3 8, 'WEIGHTED  AVERAGE  ESTIMATE  USING  FAILURE  DISCOUNTIN 
CG') 

WRITE(30,3240) 

WRITE(30,3250) 

C   COMPUTE  C. I.  FOR  WOODS  WEIGHTED  AVERAGE  ESTIMATES 

DO  3310  M=1,NPHASE 
CI  =  (1.96*EST(4,M))/SQRT(XNREP) 
CIU  =  EST(3,M)  +  CI 
CIL  =  EST(3,M)  -  CI 

WRITE(30,3260)  M,AREL(M) ,EST(3,M) ,EST(4,M) , CIL, CIU 
3310  CONTINUE 

WRITE(30,3220) 
WRITE(30,3320) 
3320  FORMATC 0 ' ,T43,' REGRESSION  ESTIMATE  USING  DISCOUNTED  FAILURES') 
WRITE(30,3240) 
WRITE(30,3250) 

C   COMPUTE  C.  I.  FOR  EXPONENTIAL  REGRESSION  ESTIMATES 

DO  3330  M=1,NPHASE 

CI  =  (1.96*REGEST(4,M))/SQRT(XNREP) 
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CIU  =  REGEST(3,M)  +  CI 
CIL  =  REGE3T(3,M)  -  CI 

V/RITE(  30,3260)  M,AREL(M)  ,REGEST(  3  ,M)  ,REGEST(4 ,M)  ,CIL,CIU 
3330  CONTINUE 

WRITE(30,3340) 
3340  FORMATC'l1 ,T59 , 'RECAPITULATION' //) 

WRITE(30,3350) 
3350  FORMAT( ' -' ,T3, 'PHASE* , Til, 'ACTUAL' ,T28, 'MEAN' ,T38  'EST' ,T53, 'MEAN' 
C,T63,'EST' ,T78,'MEAN' ,T88,'EST' ,T103,'MEAN' ,T113,fEST') 
WRITE(30,3360) 
3360  FORMATC'  ' ,T11 ,'RELIAB* ,T28,'WGT' ,T38,*STD* ,T53,'MLE' ,T63,'STD' ,T7 
C7, 'PHASE' ,T88, *STD* ,T103, 'REG' ,T113, 'STD' ) 
WRITE(30,3370) 
3370  FORMATC  ' ,T28,'AVG' ,T35 ," DEVIATION' ,T53,'W/D' ,T60, 'DEVIATION' ,T78 
C,'MLE' ,T85,* DEVIATION' ,T103,*EST' ,T110, 'DEVIATION1 ) 
WRITE(30,3375) 
3375  FORMATC  * ,T28, 'EST' /) 
DO  650  I=1,NPHASE 

WRITE( 30,3380)1, AREL( I ) ,EST( 3,1), EST( 4,1), MLEWDf 3,1) ,MLEWD( 4 , I ) , 
CMLESP(3,I) ,MLESPf4,I) ,REGEST( 3 , I) ,REGEST(4, I) 
3380  FORMATC '0'  ,T4, 13 ,T11 ,F7.  6,T26,F7.  6,T36,F7.  6,T51,F7. 6,T61,F7. 6,T76, 

CF7.6,T86,F7.6,T101,F7.  6, Till, F7.  6) 
650  CONTINUE 

GO  TO  6000 

4000  WRITE(30,4010) 

4010  FORMATC IX, //,T50, 'FIXED  PHASE  RELIABILITY  OPTION') 

WRITE( 30,4020) 
4020  FORMATC-' ,T38,' PHASE  NUMBER' ,T7 8, 'ACTUAL  RELIABILITY*) 

DO  4030  M=1,NPHASE 

WRITEC 30,4040)  M,AREL(M) 
4040  FORMATC '0' ,T41 , I2,T83 ,F8.  6) 
4030  CONTINUE 

GO  TO  5000 
6000  CONTINUE 

STOP 
END 
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APPENDIX  C.   GRAPHICAL  RESULTS 

This  appendix  contains  the  results  of  many  of  the  various  runs  done  with  the  sim- 
ulation. These  results  appear  in  tabular  format  in  the  output  file  but  have  been  reduced 
to  graphical  form  for  ease  of  understanding.  Below  each  graph  of  the  estimates  produced 
by  the  growth  models  are  graphs  of  the  standard  deviations  of  each  estimate  by  phase. 
This  appendix  is  ordered  by  reliability  growth  pattern  with  Pattern  I  first  and  the  re- 
mainder following  sequentially. 
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Figure  27.      Pattern  I,  No  Discounting 
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Figure  28. 


Pattern  I,  F  =  .25,  I  =  1 
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Figure  29.      Pattern  I,  F  -  .25,  I  =  3 
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Figure  30.      Pattern  I,  F  =  .25,  I  =  6 
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Figure  31.      Pattern  I,  F  =  .50,  I  =  3 
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Figure  32.      Pattern  I,  F  =  .50,  I  =  6 
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-*-  ACTUAL  RELIABILITY 
-D-  SINGLE  PHASE  MLE 

-V-  EXP  REG 
-  +  -  MLEFD 
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Figure  33.      Pattern  I,  F  =  .75,  I  =  3 
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Figure  34.      Pattern  I,  F  =  .75,  I  =  6 
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Figure  35. 


Pattern  I,  Lloyd,  CI  =  .8 
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Figure  36. 


Pattern  I,  Lloyd,  CI  =  .9 
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Figure  37. 


Pattern  II,  No  Discounting 
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Figure  38. 


Pattern  II,  F  =  .25,  I  =  3 
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Figure  39. 


Pattern  II,  F  =  .25,  I  =  6 
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Figure  40.      Pattern  II,  F  =  .50,  I  =  3 
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Figure  41. 


Pattern  II,  F  =  .50,  I  =  6 
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Figure  42.      Pattern  II,  F  =  .75,  I  =  3 
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Figure  43.      Pattern  II,  F  =  .75,  I  =  6 
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Figure  44.      Pattern  II,  Lloyd,  CI  =  .8 
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Figure  45. 


Pattern  II,  Lloyd,  CI  =  .9 
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Figure  46.      Pattern  III,  No  Discounting 
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Figure  47.      Pattern  III,  F  =  .25,  I  =  1 


94 


PHASE 


PHASE 


Figure  48. 


Pattern  III,  F  =  .25,  I  =  3 
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Figure  49. 


Pattern  III,  F  =  .25,  I  =  6 
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Figure  50.      Pattern  III,  F  =  .50,  I  =  3 
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Figure  51. 


Pattern  III,  F  =  .50,  I  =  6 
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Figure  52. 


Pattern  III,  F  =  .75,  I  =  3 
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Figure  53. 


Pattern  III,  F  =  .75,  I  =  6 
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Pattern  III,  Lloyd,  CI  =  .8 
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Figure  55.      Pattern  III,  Lloyd,  CI  =  .9 
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Figure  56.      Pattern  IV,  No  Discounting 
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Figure  57. 


Pattern  IV,  F  =  .10  I  =  10 


104 


-+-  ACTUAL  REUABIUTY 
-□-  MLE  SINGLE  PHASE 

-V-  EXP  REG 
-+-  MLEFD 


i 
10 


PHASE 


PHASE 


Figure  58. 


Pattern  IV,  F  =  .25,  I  =  25 
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Figure  59. 


Pattern  IV,  F  =  .25,  I  =  35 
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Figure  60. 


Pattern  IV,  F  =  .50,  I  =  60 
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Figure  61. 


Pattern  IV,  Lloyd,  CI  =  .8 
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Figure  62.      Pattern  IV,  Lloyd,  CI  =  .9 
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Figure  63. 


Pattern  IV,  Lloyd,  CI  =  .99 
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Figure  64.      Pattern  V,  No  Discounting 
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Figure  65.      Pattern  V,  F  =  .25, 1  =  1 
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Figure  66. 


Pattern  V,  F  =  .25,  I  =  3 
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Figure  67. 


Pattern  V,  F  =  .25,  I  =  6 


114 


PHASE 


Figure  68.      Pattern  V,  F  =  .50,  I  =  3 
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Figure  69.      Pattern  V,  F  =  .50,  I  =  6 
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Figure  70.      Pattern  V,  F  =  .75,  I  =  3 
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Figure  71. 


Pattern  V,  F  =  .75,  I  =  6 
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Figure  72.      Pattern  V,  Lloyd,  CI  =  .8 
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Figure  73.      Pattern  V,  Lloyd,  CI  =  .9 
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Figure  74.      Pattern  VI,  No  Discounting 
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Figure  75.      Pattern  VI,  F  =  .25,  I  =  1 
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Figure  76.      Pattern  VI,  F  =  .25,  I  =  3 
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Figure  77.      Pattern  VI,  F  =  .25,  I  =  6 
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Figure  78.      Pattern  VI,  F  =  .50,  I  =  3 
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Figure  79.      Pattern  VI,  F  =  .50,  I  =  6 
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Figure  80.      Pattern  VI,  F  =  .75,  I  =  3 
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Figure  81. 


Pattern  VI,  F  =  .75,  I  =  6 
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Figure  82.      Pattern  VI,  Lloyd,  CI  =  .8 
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Figure  83.      Pattern  VI,  Lloyd,  CI  =  .9 
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Figure  84. 


Pattern  VII,  No  Discounting 
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Figure  85. 


Pattern  VII,  F  =  .25,  I  -  1 
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Figure  86. 


Pattern  VII,  F  =  .25,  I  -  3 
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Figure  87.      Pattern  VII,  F  =  .25,  I  -  6 
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Figure  88. 


Pattern  VII,  F  =  .50,  I  =  3 
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Figure  89. 


Pattern  VII,  F  =  .50,  I  =  6 
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Figure  90.      Pattern  VII,  F  =  .75,  I  =  3 
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Figure  91.      Pattern  VII,  F  =  .75,  I  =  6 
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Figure  92.      Pattern  VII,  Lloyd,  CI  =  .8 
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Figure  93.      Pattern  VII,  Lloyd,  CI  =  .9 
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Figure  94. 


Pattern  VIII,  No  Discounting 
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Figure  95. 


Pattern  VIII,  F  =  .25,  I  =  1 
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Figure  96. 


Pattern  VIII,  F  =  .25,  I  =  3 
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Figure  97. 


Pattern  VIII,  F  =  .25,  I  =  6 
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Figure  98.      Pattern  VIII,  F  =  .50,  I  =  3 
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Figure  99. 


Pattern  VIII,  F  =  .50,  I  =  6 
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Figure  100.      Pattern  VIII,  F  =  .75,  I  =  3 
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Figure  101.      Pattern  VIII,  F  =  .75,  I  =  6 


148 


t 


ao 

d 


CO 

d 


CO 
< 

_J 

as  d 


CN 

d 


■S 


ACTUAL  RELIABILITY 
MLE  SINGLE  PHASE 

■9—   EXP  REG 

+-    MLEFD 


PHASE 


10 


Figure  102.      Pattern  VIII,  Lloyd,  CI  =  .8 
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Figure  103. 


Pattern  VIII,  Lloyd,  CI  =  .9 
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